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ABSTRACT 



Recent research suggests that principals of effective schools exhibit 
Instructional leadership behavior such as participating In teachers' staff 
development programs and observing and giving feedback on teachers' 
Instruction. Because that research employed descriptive and correlational 
research designs, however, It Is unclear whether Improved school 
effectiveness Is caused by the principal's Instructional leadership. The 
present experiment was conducted to determine whether systematically 
manipulating principals' Instructional leadership behavior would lead to 
positive changes In teachers' instruction and students' achievement. 

Fifty-three fourth- and fifth-grade teachers were assigned to three 
treatment conditions. Active Mathematics Teaching, a staff development 
program developed by Good and Grouws, was used as the Intervention. This 
program trains teachers In an Instructional model that Involves assigning 
homework, conducting continual review, emphasizing whole group development 
Instruction, and monitoring and checking seatwork. 

Teachers In the principal Involvement condition participated In this 
program along with their principals, in addition, the principals received 
Instructional leadership training and conducted observations and conferences 
with their teachers. Teachers In the regular Inservlce condition 
participated In the program, but their principals were not involve'' i any 
way. In the control condition, neither the teachers nor their principals 
participated in the program. 

Teachers' math lessons were observed Immediately before and after the 
staff development program, and again three months later. Also, their 
students were administered curriculum-referenced and nationally standardized 
mathematics tests before and three months after the program. 

Both trained groups of teachers outperformed the untrained group In 
Implementing the program's Instructional model In the post lesson (p».001) 
and delayed lesson {p».O01). Both trained groups also had a lower percentage 
of off-task students In the post lesson {p«.10) and delayed lesson {p«.05). 
Students of teachers In both trained groups made slightly greater gains than 
control group students on the achievement tests. 

The two trained groups did not differ In Implementation of the 
Instructional model In the post lesson, but the principal Involvement group 
had a higher Implementation level (p».07) In the delayed lesson. The two 
groups did not differ In percentage of off-task behavior In either lesson, 
but this was not an explicit objective of the staff development program. 
Students of teachers In the principal Involvement group outgalned students of 
teachers In the regular Inservlce group {p«.07) on the curriculum-referenced 
test. The regular Inservlce group outgalned the principal Involvement group 
(p^.lO) on the computation subtest of the standardized measure. 

Taken together, the results demonstrate that Involving principals 1n a 
staff development program Increases Implementation of the program's 
objectives. If their Instructional leadership role Is activated, principals 
can (1) selectively direct teachers' attention to particular Instructional 
improvement objectives, and (2) help teachers maintain the improvement over 
time. The study also found that district administrators play a low-profile, 
but central role in istaff development program Implementation. 
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problem investigated here. Bob Mattson provided continuing dialogue about 
how our study relates to larger Issues of educational policy. Jane Arends 
contributed efficient and insightful administrative support to the project. 
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the preparation of the report. Terri Williams and Sissel Lemke typed the 
report. We thank all of you for you " help. 

Meredith Gall, professor of education, was one of three project 
co-directors. He prepared the technical proposal for the project, managed 
the collection of classroom observation data, and wrote the final report with 
the exception of Chapter 3. Any Inaccuracies in the report are his 
responsibility alone. Glen Fielding, associate professor of education, was 
another project co-director. He designed many of the experimental 
procedures, provided continuing liaison to the schools, implemented the 
Instructional leadership component of the experiment, and wrote most of 
Chapter 3. Del Schalock, research professor of education and assistant dean, 
was the third project co-director. He made essential contributions to the 
design of the experiment and was largely --esponslble for the recruitment of 
the district sites. He also co-founded the Valley Education Consortium and 
developed the approach to curriculum- referenced testing used in the 
experiment. 

Sandy Charters, professor of education, brought order to the data by 
conceptualizing and executing most of the statistical analyses presented in 
Chapter 4. In addition, he contributed to the final report by interpreting 
the meaning and significance of the statistical results. Jerzy Wilczynski, 
assistant professor of computer science, created the computer data files used 
In all of the analyses, and he executed many of the district-level analyses 
reported in the appendix. Miguel Perez, a doctoral student in educational 
psychology, also contributed to this effort. 
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CHAPTER 1 
RESEARCH PROBLEM AND OBJECTIVES 
Statement of Purpose 

The purpose of the present study was to determine whether the effectiveness 
of a staff development program for elementary teachers can be Improved by 
training the teachers' principals In Instructional leadership skills related to 
the program. Although Instructional leadership Is considered an Important 
characteristic of good school administrators, few attempts have been made to 
study Its effects by experimental manipulation. 

A related purpose of the study was to replicate previous research on a spe- 
cific staff development program called Active Mathematics Teaching. This 
program was developed and.valldated by Tom Good and Doug Grouws at the 
University of Missouri (Good, Grouws, and Ebmeler 1983). Their research 
demonstrated that the program Increased teachers' use of active teaching beha- 
viors and, subsequently, Improved students' math achievement. 

Ine experiment reported here was done under the auspices of the Center for 
Educational Policy and Management (CEPM) at the University of Oregon. One of 
CEPM's missions Is to discover how staff development can be used as an Instru- 
ment of policy to bring about school Improvement (Duckworth 1983). The present 
study Is related to that mission. 

The Problem 

Staff development continues to be of Interest as a strategy for improving 
the use of human resources (teachers, teacher aides, pr^incipals, and others) In 
schools. Many educators are Involved directly or indirectly in staff develop- 
ment (Joyce, »<owey and Yarger 1976), and there is substantial investment in it 
by universities, school systems, and individual educators. For example, a sur- 
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vey of Oregon school districts several years ago (Schalock 1977) found that 
typically 3 to 5 percent of school district budgets Is allocated to staff 
development although It Is usually not Identified as a specific line Item. 

Despite continuing Interest In staff development. It remains a poorly 
understood phenomenon. A previous project by CEPM researchers (Gall, Halsley, 
Baker, and Perez 1982) found that staff development Is prevalent In school 
districts, but that It lacks many of the characteristics Identified by research 
as effective. Teachers receive almost two we''''s of Inservlce per year but It 
consists mostly of one-shot activities dispersed over a wide range of unrelated 
goals. 

The use of staff development as an Instrument of policy for school Improve- 
ment has not been clearly articulated. As a start toward formulating such a 
policy. It Is helpful to think of staff development as having four purposes: 

1. the personal professional development of teachers; 

2. the credential ling of teachers for role positions; 

3. the Induction of new teachers Into the profession; 

4. school Improvement. 

These purposes are not necessarily mutually exclusive. A particular staff deve- 
lopment program can serve several purposes. In practice, though, staff develop- 
ment programs are either Instruments of policy to promote the development of the 
Individual teacher (the first three purposes) or they are Instruments of policy 
to promote school Improvement. This policy distinction Is evident In the cycli- 
cal Staff Development/School Change model of Inservlce education developed by 
Miller and Wolf (1979) and In the separate models of Inservlce education to meet 
school system needs and Individual teacher needs developed by Campbell (1981). 
The present study was Intended to develop knowledge about the use of staff 
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development as an Instrument of policy to promote school Improvement. The pri- 
mary unit of sampling, Intervention, and analysis was the Individual school. An 
experiment was done to determine whether a particular staff development program 
could Increase the capacity of a school's teachers and principal to Improve the 
basic skills achievement of students. 

The connections between staff development and school effectiveness are for 
the most part assumed rather than empirically validated. Our previous research 
(Gall et al. 1982) suggests that the causal chain from staff development acti- 
vity to teacher change, and then to student gains. Is poorly articulated in 
practice. There Is experimental evidence, however, that this sequence of 
effects Is possible. 

This line of research Includes a recent set of four experiments reported by 
Good and Grouws (1979); Andferson, Evertson, and Brophy (1979); Crawford et al. 
(1978); and Stalllngs (1980). (These experiments are summarized in Gage and 
Glaconia 1981, and in Mohlmdn, Coladarcl, and Gage 1982). Each of the experi- 
ments was successful in demonstrating positive effects of a staff development 
program on teacher instructional performance and student achievement in basic 
skills. The content of the staff development programs Involves organized sets 
of behavioral ly-defined instructional skills that were validated against student 
achievement criteria in prior correlational research. These skills collectively 
have been called a "direct instruction" model (Rosenshine 1976). 

A staff development program that was tested in one of the experiments (Good 
and Grouws 1979) was used in the present study. This staff development program 
was selected for several reasons. First, it produced clear, significant effects 
on student achievement in three experiments (GoOd arid Gfouws, 1979; Good et al. 
1983). Second, teachers reacted favorably to this program (Keziah 1980; Andros 




and Freeman 1981). Th1rd» the school districts that were recruited for the pre- 
sent study had Identified mathematics — the subject of the Good and Grouws 
inservlce program — as a priority for school Improvement. And fourth, the 
direct Instruction skills covered In the Good and Grouws program are not tied to 
a specific mathematics curriculum. 

An Important problem In research on staff development Is whether variations 
in format of staff development programs affect their outcomes. Relatively 
little Is known about which variations are most effective. The variation of 
Interest In this study was whether Involvement of school principals In staff 
development programs for teachers contributes to the programs'' S^ectlveness, 

There Is currently much Interest In t^e role of the principal as Instruc- 
tional leader. Recent reviews of research on Ins^* ctl'. ^aT leadership (Sweeney, 
1982; DeBevolse 1984) have concluded that the principal's leadership behavior 
has a significant effect on students' academic achievement. This research con- 
sists entirely of correlational and case studies, however. Experiments are 
needed to determine the causal properties of principal leadership behaviors and 
to validate procedures for Increasing principal leadership capability. 

It seems reasonab.e to hypothesize that a school principal might enhance or 
undo the Intended effects of a staff development program designed to Increase 
basic skills achievement of students. On the positive side, the principal can 
engage In such behaviors as demonstrating approval of the program's goals, 
rewarding teachers for Implementation of the program's objectives, securing 
resources, and facilitating needed organizational change. On the negative side, 
the principal can weaken the inservlce program by such behaviors as refusing to 
acknowledge its presence, encouraging teachers to pursue other priorities, and 
failing to provide needed resources. These positive and negative behaviors seem 
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Important, but they have been little studied through experimental research. A 
major purpose of the present study was to determine whether school principals 
can be trained In positive leadership behaviors specific to staff development 
programs for teachers, and whether such training enhances the effects of staff 
development programs on teachers and students. 

The following hypotheses were tested In the experiment: 

1. Teachers who participate In the staff development program (with or 
without principal Involvement) will use active teaching techniques 
more frequently than teachers who do not participate In the program, 

2. Teachers who participate In the staff development program with 
principal Involvement will use active teaching techniques more 
frequently than teachers who participate In the program without 
principal Involvement. 

3. Students of teachers who participate In the staff development program 
(with or without principal Involvement) will earn higher scores on 
mathematics achievement tests than students of teachers who do not 
participate In the program. 

4. Students of teachers who participate In the staff development program 
with principal Involvement will earn higher scores on mathematics 
achievement tests than students of teachers who participate In the 
program without principal Involvement. 

5. Students of teachei's who participate In the staff development program 
(with or without principal involvement) will exhibit less off-task 
behavior during Instruction than students of teachers who do not 
participate in the program. 

6. Students of teachers who participate in the staff development program 
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with principal Involvement will exhibit less off-task behavior than 
students of teachers who participate In the program without principal 
Involvement. 
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CHAPTER TWO 
REVIEW OF LITERATURE 
Three areas of research are relevant to the present study: research on the 
effects of staff development; research on variations In staff development; and 
research on principals' Instructional leadership. Each area will be discussed 
as It relates to the purpose of the study. 

Effects of Staff Development 
The Intent of staff development (sometimes called "Inservlce education") Is 
to increase teacher effectiveness. Bruce Joyce and his colleagues have defined 
Inservlce education as the "form,^l and Informal provisions for the Improvement 
of educators as people, educated persons, and professionals, as well as In terms 
of the competence to carry out their assigned roles" (Joyce, Howey and Yarger 
1976, p. 6). 

Staff development Is a large, labor-intensive effort that Involves the 
expenditure of substantial school system resources. In 1975 It was estimated 
that "...there may be as many as a quarter of a million persons in the United 
States who engage as Instructors in some form of ISTE (Inservlce teacher 
education) activity— this Is about one Instructor for every eight teachers" 
(Joyce et al. 1976, p. 6). At that time, too, about half of American teachers 
held a master's degree, representing a considerable Investment In Inservlce edu- 
cation beyond initial certification. But despite this considerable investment, 
the research basis for the effectiveness of staff development is virtually 
nonexistent. Most of the research consists of evaluating the immediate effects 
of staff development on teacher attitudes, knowledge, and behavior (Showers 
1982). 

An Important new line of research is the small but growing number of studies 
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that have Investigated, through controlled experimentation, the effects of staff 
development on teacher effectiveness— defined as capability to bring about 
Improvements In student performance and basic skills achievement. These studies 
are recent; most of them were completed In the last five years: Anderson, 
Evertson, and Brophy (1979); Coladarcl (1980); Crawford et al. (1978); three 
experiments by Good, Grouws, and Ebmeler (1983); and Sta111ngs U980). In each 
study a group of Inservlce teachers received the experimental training program 
while a control group of teachers continued their regular activities* Following 
the training phase, researchers observed the students of both groups over a 
period of time to determine training effects on students' academic achievement. 
Because these experiments are central to the development of a model of effective 
staff development, they are discussed In detail here. 

With the exception of the Coladarcl study, ea''h experiment demonstrated that 
teachers who received the Inservlce program learned to use particular Instruc- 
tional methods more frequently than teachers who did not receive tho training. 
More Importantly, the students of the trained teachers scored higher on achieve- 
ment tests of basic skills than control group teachers (again, with the excep- 
tion of the Coladarcl study). For example, students of the experimental teachers 
In Good and Grouws' fourth-grade experiment Increased from a percentile of 26.57 
to 57.58 on the SRA Mathematics Achievement Test over a two and one-half month 
period. The control group (the whole-dass Instru'ctlon subsample) made a 
substantially smaller gain — from the 25th percentile to the 43rd percentile on 
the same test. 

Table 2-1 summarizes the methods used In the experiments. Examining com- 
monalities In the four successful experiments, we find that each of the Inser- 
vlce programs Involved at least two meetings- (The "minimal" group In Crawford 
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Table 2-1 

Instructional Process Used In Basic Skills Inservlce Programs 

1. Anderson, Evertson, and Brophy (1979) 

Project staff met with teachers to discuss the study. Teachers then read a 
33-page manual describing 22 research- vail dated principles of reading group 
Instruction, and took a short quiz on It. Teachers met once again with pro- 
ject staff to discuss the manual. One subgroup of these teachers was 
observed for their Implementation of the principles throughout the school 
year. Another subgroup was not observed. (End-of-year student achievement 
In the two trained groups did not differ from each other). 

2. Coladarcl (i980) 

Five training packets (the same as In the Crawford and Stal lings experiment) 
were mailed to experimental group teachers. The teachers received one 
packet per week, and responded to a test and questionnaire on it. 

3. Crawford and Stal lings (1978) 

The "minimal" training group received one training manual and self- 
ddmlnlstered test per week for five weeks. The "maximal" group received the 
san?e manuals and tests, and also came to a two-hour meeting with project 
stiff each week. In these meetings the teacher discussed, practiced, and 
studied the techniques; engaged In role-playing exercises; and viewed 
videotapes of a "model" teacher performing the behaviors. (End-of-year stu- 
dent achievement was higher for "maximal" group). 

4. Good and Grouws (1979) 

Teachers attended an Introductory 90-m1nute meeting and then read a 45-page 
manual of research- val 1 dated principles of mathematics Instruction. Two 
weeks later teachers attended another 90-m1nute meeting In which project 
staff responded to their questions and concerns. 

5. Stal lings (1980) 

Each teacher was observed for three days, then given a quantitative summary 
cf the observation as feedback to help change his/her Instruction to conform 
tv-^ research-validated specifications. Teachers also attended four two-hour 
v/o-kshops over a two-and-one-half month period. 
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and Stalllngs' study did not Include aiy niaetlngs, with resulting lower end-of- 
year achievement scores relative to the "maximal" group). Another common 
feature across the studies Is the use of brief manuals to describe the desired 
behaviors. 

Teacher behavior was observed and critiqued In two of the studies. 
Stalllngs* teachers were observed In their classrooms and given both a qualita- 
tive and quantitative summary of the results. Crawford's "maximal" group 
teachers were observed In role-playing exercises during meetings. Teacher beha- 
vior was observed In one of Anderson's trained groups, but the observations were 
not shared with the teachers. 

It appears that extensive skill training of the type used in microteaching 
programs (e.g., Borg, Kelley, Langer, and Gall 1970) is not necessary in a 
basic skiljs inservice program. The critical elements appear to be (a) the 
opportunity to study manuals that present research-validated principles of 
instruction, and (b) the opportunity to discuss these principles in meetings 
with an inservice trainer and other teachers at the same grade level. The value 
of supplementing these process elements with classroom observation and feedback 
has not yet been clearly established. 

The results of this research suggest that teacher effectiveness in basic 
skills instruction can be Increased by a relatively simple staff development 
process. It should be noted, though, that these studies extended over a period 
of no more than a single school year. Also, the training programs were not suc- 
cessful for all teachers. Training processes not used in the five experiments 
described above may produce more sustained effects, and effects for more 
teachers, than those observed in the experiments. For example, the school 
principal's instructional leadership may help some teachers to overcome their 
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resistance to Implementing the program's objectives. The effective principal 
may also be ♦b.lg, to provide continuing leadership to help all teachers su:ta1n 
changes In their classroom behavior over substantial periods of time. 

Good and Grouws' Inservlce Program 
Of the four successful Inservlce programs, the one chosen for the present 
experiment was developed by Thomas Good and Douglas Grouws (Good et al. 1983). 
Reasons for selecting this program were described In Chapter 1. The key 
instructional behaviors covered In the program's training manual are presented 
In Table 2-2. 

The behaviors were derived from previous correlational research on the rela- 
tionship between Instructional processes and student achievement processes In 
regular classroom settings (Good and Grouws 1977). Together the behaviors form 
an Instructional system having these characteristics: 

1. Instructional activity is initiated and reviewed in the context of 
meaning. 

2. Students are prepared for each lesson stage to enhance involvement 
and to minimize errors. 

3. The principles of distributed and successful practice are built 
into the system. 

4. Active teaching is demanded, especially in the developmental 
portion of the lesson. (Good and Grouws 1981, p. 4). 

These characteristics are similar to those of the other instructional systems 
tested in the experiments described above. 

The effectiveness of the Good and Grouws training program has been tested 
In three experiments by the developers (summarized in Good et al. 1983) and in 
another experiment by independent investigators (Slavin and Karweit 1984). In 
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Table 2-2 

Good and Grouws' Techniques for Mathematics 
Instruction In Elementary School 



Dally Review (first 8 minutes except Mondays) 

(a) review the concepts and skills associated with the homework 

(b) collect and deal with homework assignments 

(c) ask several mental computation exercises 

Development (about 20 minutes) 

a) briefly focus on prerequisite skills and concepts 

b] focus on meaning and promoting student understanding by using 
lively explanations, demonstr^ttlons, process explanations. 
Illustrations, etc* 

(c) assess student comprehension 

(1) using process/product questions (active Interaction) 

(2) using controlled practice 

(d) repeat and elaborate on the meaning portion as necessary 

Seatwork (about 15 minutes) 

(a) provide uninterrupted successful practice 

(b) momentum— keep the ball roll1ng--get everyone involved, then 
sustain Involvement 

(c) alerting— let students know their work will be checked at end of 
perl od 

(d) accountability— check the students' work 

Homework Assignment 

(a) assign on a regular basis at the end of each math class except 
Fridays 

b) should Involve about 15 minutes of work to be done at home 

c) should Include one or two review problems 

Special Reviews 

(a) weekly review/maintenance 

(1) conduct during the first 20 minutes each Monday 

(2) focus on skills and concepts covered during the previous 
week 

(b) monthly re view/ maintenance 

(1) conduct every fourth Monday 

(2) focus on skills and concepts covered since the last 
monthly review 
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the first experiment, one group v)f fourth-grade teachers received training by 
attending two meetings and reading a manual. The control group of teachers did 
not receive the traininc,. The training closely paralleled the training used in 
the present experiment: the manual was the same, and there were two meetings. 

The second study Involved a similar experimental design, but was done with 
sixth-grade teachers. The training format was similar to that of the first 
experiment, except that a verbal problem-solving manual was added to the origi- 
nal training manual. This new manual covers techniques for using verbal problems 
without numbers, writing verbal problems, estimating the answer, reading verbal 
problems, and writing open sentence problems. 

The third experiment maintained the same experimental design as the first 
two studies, but was done with Junior high teachers (eighth grade). Also, the 
training manual included a11 of the material from the manual used in the second 
study, plus two supplements: Teachers* Manual Addendum for Junior High Work, 
and a new Procedural Summary for the Verbal Problem Solving Manual. 

The effects of Good and Grouws' training program on students' mathematics 
achievement are shown in Table 2-3. Comparisons favored the trained group for 
each measure in each experiment, although the effect was not always statisti- 
cally significant. 

Effects of the training program on teachers' behaviors were systematically 
assessed in the fourth-grade experiment. Implementation results are shown in 
Table 2-4, The trained teachers conformed more closely to the instructional 
model than did the control teachers. An exception is that the training 
apparently had little or no effect on the amount and quality of time spent on 
developmental Instruction (variables 2, 7, 8, and 9). 

Table 2-4 also shows the correlations between teachers' scores on the 
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Table 2-3 



Summary of Student Achievement Effects Observed 
In Three Experiments by Good and Grouws (1981) 



(Raw Scores) 


Trained Teachers 
Pre Post 


Pre 


1 c€ivncr> 

Post 


b1 
P 


Fourth-Grade Experiment 
dKA Hatn lest M 

(SD) 


11.94 
(3.18) 


19.95 
(4.66) 


12.84 
(3.12) 


17.74 
(4.76) 


•002 


Sixth-Grade Experiment 
SRA nath lest M 

(SD) 


25.03 
(5.0) 


28.96 
(4.8) 


26.80 
(4.1) 


29.65 
(3.7) 


NS 


Math Problem-Solving 

1 est . \ 

(SD) 




14.90 
(2.0) 




14.71 
(1.6) 


.10 


Junior High Experiment 
SRA Computation 
Subtest^ Adjusted M 




29.84 




28.86 


NS 


SRA Math Problem^ 
Solving Test 

Adjusted M 




21.98 




20.83 


.03 



1. Probability value based on analysis of covarlance with appropriate pretest 
as covarlate. 

2. This test was administered post-treatment only. SRA pretest was the covarlate. 

3. These tests were administered post-treatment only. A special math pretest 
was the covarlate. 
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Table 2-4 



Mean Occurrence (In percentages) of Implementation Behaviors 
In Fourth-Grade Experiment and Correlation of the Variables 
and Residual 1 zed Math Achievement Scores » Good and Grouws (1981) 

Variable Treatment Control p Correlation p 

1* Did the teacher conduct 

review? 91X 62% .0097 .37 .04 

2. Did development take place 

within review? 51% 37% .16 .10 .57 

3. Old the teacher check 

homework? 79% 20% .0001 .54 .001 

4. Old the teacher work on 

mental computation? 69% 6% .001 .48 .005 

5. Old the teacher sunmarlze 

previous day's materials? 28% 25% .69 .20 .26 

6. There was a slow transition 

from review. 7% 4% .52 -.02 .91 

7. Did the teacher spend at least 

5 minutes on development? 45% 51% .52 -.08 .65 

8. Were the students held account- 
able for controlled practice 

during the development phase? 33% 20% .20 .12 .50 

9. Did the teacher use demonstra- 
tions during presentation? 45% 46% .87 -.15 ,41 

10. Did the teacher conduct 

seatwork? 80% 56% ,004 .27 .13 

11. Did the teacher actively 
engage students in seatwork 

(first U/Z minutes)? 71% 43% .003 .32 ^7 

12. Was the teacher available to 
provide immediate help to 
students during seatwork 

(next 5 minutes)? 68% 47% .02 .28 ,11 

13. Were students held accountable 
for seatwork at the end of 

seatwork phase? 59% 31% .01 .35 ,05 

14. Did seatwork directions take 

longer than one minute? 18% 23% .43 -.02 .92 

15. Did the teacher make homework 

assignments? 66% 13% .001 .49 .004 



^ „ 21 



Instructional behaviors and the residual Ized achievement gains of their students 
on the SRA Mathematics Test. These results suggest that mental computation 
(variable 4) and homework (variables 3 and 15) are the most critical behaviors 
In Good and Grouws* Instructional system for promoting student achievement. 

Data collected by Good and Grouws In the three experiments Indicate that the 
trained teachers generally were very pleased with the training program. 
Similarly favorable reactions of trained teachers were also reported by Kezlah 
(1980) and by Andros and Freeman (1981). 

Slavin and Karwelt (1984) recently completed two experiments Involving Good 
and Grouws* program. In the first experiment teachers In grades 4-6 were ran- 
domly assigned to three treatments: Good and Grouws' program; training In Good 
and Grouws' program but with the request that each teacher ability-group his or 
her students within the class; and training In an Individualized Instruction 
model involving cooperative learning by students. Results revealed no treatment 
differences on the CTBS subtest of Concepts and Applications. The ablllty- 
grouped and Individualized treatments, however, were both substantially superior 
to the regular Good and Grouws' program on the CTBS subtest of Computations. 

The second experiment involved the same design, with the addition of an 
untreated control group. The treatment effects observed In the first experiment 
were also found in this experiment. Also, It was found that each of the trained 
groups (Including teachers receiving the regular Good and Grouws' program) 
excelled the control o'-oup on the Computations subtest but not on the Concepts 
and Applications suote;>c. 

Inservlce Variations 
The literature on Inservlce education. Including thousands of entries In 
ERIC and a book-length bibliography (Collins et al. 1979), emphasizes its 
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complex nature. Content, training methods, purposes, governance, and organiza- 
tional context must be considered. The effectiveness of a particular Inservlce 
variable In Improving teacher and student performance depends on how other 
aspects of the Inservlce program have been constituted. For example, variations 
In Inservlce training methods may have little effect when the criterion Is stu- 
dent achievement gains averaged across teachers for a single school year. 
Training variations may have a substantial effect, though, when the criterion 1s 
student achievement gains for as many teachers as possible In a school sita and 
sustained over a multi-year period. 

In a previous study, one of the present researchers and his colleagues 
(Gall, Halsley, Baker and Perez 1982) developed a staff development model that 
Includes 29 dimensions. It Is an elaboration and restructuring of the model 
developed by the staff of the Inservlce Teacher Education Concepts Project 
(Joyce, Howey and Yarger 1976). The primary source for the elaborated model was 
the recent conceptual and empirical work on curriculum Implementation and school 
Improvement (Hall and Loucks 1980; Berman and McLaughlin 1978). Table 2-5 pre- 
sents the list of staff development dimensions compiled by Gall and colleagues. 

The model proved useful in designing the present study because 1t suggested 
features that should be Included In a treatment or controlled across treatments. 
For example, the present experiment did not manipulate presence/absence or type 
of readiness activities (dimension 13). Research on curriculum Implementation 
however, suggests that readiness activities are Important to the success of a 
new school program (Including Inservlce programs). Thus, appropriate readiness 
activities were Included In each of the experimental treatments so that the 
absence of this factor did not depress the effectiveness of the staff develop- 
ment program. Similarly, such factors as scheduling and training site con- 
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Table 2-S 

Dimensions for Analyzing Insemce Education 

A. Teacher Objectives of an Inservlce Activity 

1. Target competencies 5, Expected level of competence 

2. Side- or long-term effects 6. Measurement of teacher competence 

3. Operational Izatlon 7. Rationale 
4« Complexity 



B. Student Objectives of an Inservlce Ac 

8. Needs assessment 11. 

9. Target objectives 12. 
10, Side- or long-term effects 



vlty 

♦V 

Measurement of student objectives 
Expected level of proficiency 



C. Delivery System 

13. Readiness activities 

14. Instructional process 

15. Maintenance/monitoring 

16. Training site 



17. Relevance of content 

18. Trainer 

19. Scheduling 

20. Cost 



D. Organizational Context 

21. Purpose for participation 23. Concurrent organizational changes 

22. Inservlce cohorts 



E. Governance 

24. Governance structure 27. Incentives 

25. Teacher participation In 28. Sanctions 
governance 29. Rationale 

25. Participation 
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ditlons were designed In accordance with the best knowledge available about 
teacher preferences. 

The Good and Grouws Inservlce program specifies some of the key dimensions 
of the model. For example, the program specifies: target competencies (no. 1), 
operatlonalizatlon of competencies (no. 3), measurement of teacher competence 
(no. 6), rationale for teacher objectives (no. 7), target objectives (no. 9), 
and measurement of student objectives (no. 11). 

The principal's Instructional leadership Is subsumed in the model shown In 
Table 2-5 under "concurrent organizational changes" (dimension 23) that occur 
during an Inservlce program. When an Inservlce program Is used for school 
Improvement It seems likely that various organizational conditions (e.g., prin- 
cipal leadership, curriculum revision, staff changes) can support or Interfere 
with the program's objectives. Of these conditions, principal leadership seems 
the most critical to the success of a staff development program directed toward 
school Improvement. The next section reviews the research evidence in support 
of this proposition. 

Instructional Leadership of Principals 

DeBevoise (1984) concluded from her review of the literature that there is a 
lack of consensus about the meaning of the term "instructional leadership." 
According to some educators, instructional leadership refers to any function 
that promotes effective operation of a school. Other educators define instruc- 
tional leadership in terms of more specific functions such as teacher super- 
vision, management of school change efforts, and clarification of school goals. 

The present study used a restricted definition of instructional leadership. 
Instructional leadership was defined here as the principal's demonstration of 
skills and behaviors that Increase the effectiveness of a staff development 
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program for the school's teachers. In the case of Good and Grouws* program, 
effectiveness Is defined In terms of (a) teachers' use of an active teaching 
model for the mathematics curriculum, and (b) students' galii In mathematics 
achievement. According to these definitions, a principal demonstrated instruc- 
tional leadership In the present study whenever he or she engaged in behaviors 
that helped teachers to use the active teaching model more skillfully and more 
often, and whenever he or she engaged In behaviors that helped students to 
Improve their mathematics achievement. 

Research on Instructional leadership has become widespread In recent years. 
Most Investigations have Involved comparisons of more effective and less effec- 
tive schools, where effectiveness has been defined In terms of student achieve- 
ment gains aggregated to the s-hool level. Reviews of these Investigations have 
been done by Bossert, et al. (1981), DeBevolse (1984), Pitner (1981), Shoemaker 
and Fraser (1981), and Sweeney (1982). 

Table 2-6 summarizes effective Instructional leadership characteristics and 
behaviors based on available research and on a recent comprehensive review of 
research (Leithwood and Montgomery 1982). Most of these characteristics and 
behaviors are stated at a high level of generality. However, they provided use- 
ful suggestions for designing the Instructional leadership component of the 
principal Involvement treatment In the present experiment. For example, several 
studies (Kean et al. 1979; New York State Office of Education 1974; Welllsch 
et al. 1978) found that principals of effective schools regularly observed and 
gave feedback on teachers* Instruction. Also, effective principals attend 
teachers' Inservlce sessions (Leithwood and Montgomery 1982). Based on these 
research findings, principals In the principal involvement treatment were given 
training and directions to attend Good and Grouws' program and then to observe 
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Table 2-6 

Characteristics and Behaviors of Effective Principals 
Identified in Correlational Studies 



Study 



More effective principals; 
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1. Austin (1978) 



2. Brookover & Lezotte 
(1979) 



3. Edmonds (1979) 



4. Kean et al. (1979) 



5. Madden et al. (1976) 

6. New York State Office 
of Education (1974) 

7. Venezky & Winfleld 
(1979) 

8. Weber (1971) 



9. Welllsch et al. (1978) 



10. Leithwood and 
Montgomery (1982)1 



- participate more fully In Instruction 

- have high expectations of themselves, their 
teachers, and their students 

- are oriented toward cognitive rather than 
affective goals 

- are assertive In their Instructional 
leadership 

- assume responsibility for evaluating the 
achievement of basic objectives 

- are disciplinarians 

- communicate effectively with their teachers 
and respond to teacher difficulties 

- make frequent observations of reading 
classes 

- are reading specialists 

- have more Impact on educational decision making 

- engage In positive Interaction with teachers 

- make frequent Informal classroom observations 

- have a strong desire for academic achievement 
and transmit this desire to staff and pupils 

- are clearly Identifiable Instructional leaders 

- lead their school's beginning reading program 

- express a strong concern about the 
Instructional program 

- regularly review and discuss teachers' 
performance 

- participate more often In Instructional 
decisions 

- place the academic achievement and happiness 
of students first In their priorities 

- articulate high expectations for teachers 

- hold regular and frequent staff meetings 

- attend Inservlce sessions provided for 
teachers 

- support and provide staff development for 
teachers 

- monitor student progress closely 

- facilitate communication between the school 
and the community 
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Table 2-6 Continued 



U. Sweeney (1982)^ - give high priority to activities, Instruction, 

and materials that foster academic success 

- take part In Instructional decision-making 

* ensure that the school's climate Is conducive 
to learning 

- monitor student achievement on a regular basis 

- Interrelate course contents sequences of 
objectives, and materials In all grades 

- support teachers' participation In Inservlce 
programs 



A review of research on characteristics of effective principals. 
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their teachers as they worked on Implementing the active teaching system In 
their classrooms. 

The research on Instructional leadership of principals summarized in Table 
2-6 was i)ased on descriptive and correlational designs. No studies have been 
reported 1n which Instructional leadership behaviors were experimentally manlpu 
lated to determine their effect on teachers' classroom performance and on stu- 
dent achievement. The present study makes a departure from prevtous research 
because It Includes manipulation of Instructional leadership behavior as a pri- 
mary feature of the experimental design. 
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CHAPTER THREE 
RESEARCH METHODOLOGY 



Research Design 

The research hypotheses stated In Chapter 1 were tested using a pre<-post 
experimental -control group design. Schools were nonrandomly assigned to 
three treatment groups. The research design Is shown as follows: 
Op Og, O3 Xj^ Oj, Og, O3 
0^. O2, O3 X2 Oj, Og, O3 
Oi, O2, O3 X3 Oj, Og, O3 
where: 

0. > observational measure of teacher behavior and student 

* off- task behavior 

02 * curriculum-referenced test of student math achievement 

03 " nationally standardized test of student math achievement 

X. « Good and Grouws' inservlce program without principal 

* Involvement 

Xp * Good and Grouws' Inservlce program with principal 
1 nvol vement 

X3 • control condition Involving the absence of training In 
Good and Grouws' program 

« 

Fifteen schools were assigned to the three treatment conditions before 
pretralning observations began. All participating classrooms In a school 
were assigned to the same treatment condition. Teachers In the Inservlce 
program treatments participated In two training sessions scheduled during the 
period January 31 - February 18, 1983. Principals In the principal 
Involvement treatment attended the same two sessions and two additional 
leadership sessions, one at the end of February and the other at the end of 
April. Teachers In the principal Involvement treatment participated In 
follow-up activities with their building principals spaced over several 
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months tfttr th« two trtlning mslons. 

Classroom observation of teacher behavior and student off- task behavior 
In a math lesson occurred at three points 1n time* The first lesson was 
observed during the week of January 24, 1983, which was prior to the staff 
development program; the next lesson was observed during the two week period 
of March 8 March 18, 1983, following the staff development program; and the 
final lesson was observed during the two-week period of May 3 - May 13, 1983. 
A few teachers who were unavailable within the two-week period were observed 

4 

the week following. 

The curriculum-referenced test was administered In January 1983, and 
then again In May 1983. The nationally standardized test was administered In 
one district In October 1982 and then again In May 1983. In the other two 
districts the test was administered In May 1982 and then again In May 1983. 

Recruitment of Sample 

Recruitment of Districts 

Three districts In western Oregon participated In the experiment. Each 
of the participating districts was selected because of the commitment 
expressed by district administrators to Increasing student achievement In 
elementary mathematics, and because of the long-term program Improvement and 
staff development activities that each had sponsored. The study was viewed 
by district personnel as an opportunity to extend Instructional change 
efforts already underway. 

One participating district (designated District 1 hereafter) was 
previously engaged In an extensive Instructional Improvement effort designed 
to promote teachers' use of a model of Instruction called "Elements of 
Effective Instruction" and commonly referred to as "Instructional Theory Into 
Practice" (hereafter the model will be referred to as ITIP) (Hunter 1976). 
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Tht district had orgtnlztd • str1«s of coursM ovtr itvtrAl ytars for 

teachers In using the model. Courses for principals also had been offered to 
enable them to use the model as a basis for Instructional supervision. 

Administrators In this riistrlct regarded the Good and Grouws' model as 
complementary to the ITIP approach. The models have several Important 
features, In common. For example, both emphasize the Importance of having the 
teacher provide structured, carefully monitored practice In using a skill 
before permitting students to practice It Independently. Active teaching, 
however, focuses more specifically on mathematics Instruction, whereas ITIP 
applies to teaching across subject areas. District personnel perceived the 
studiy as an opportunity for teachers to apply the more general training In 
Instructional skills that the district had provided previously to a specific 
curriculum area. 

During 1982-83 two of the districts (designated Districts 2 and 3 
hereafter) were field testing an elementary mathematics program that teachers 
from the districts had helped to develop. This program, which Is described 
In greater detail later In the chapter. Includes learning goals for students, 
test Items for monitoring student progress toward goal attainment, and 
guidelines for administering the program. The program does not provide, 
however, a structure for organizing and managing dally lessons at the level 
of specificity In Good and Grouws' "active teachlnj^" model. District staff 
wished to determine whether training In the Instructional model would enhance 
teachers' capacity to foster students' achievement of the learning goals 
specified In the districts' new mathematics program. 

Recruitment of Schools Within Districts 

All elementary schools In the three districts that contain at least one 
fourth grade and one fifth grade classroom were Included In the sample. 
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Sevtral schools In two of tha districts contalntd cUssrooms with children at 
many different grade levels. These were small schools In Isolated areas 
within the districts, and appeared to be unrepresentative of elementary 
schools In general* They were excluded from the sample. The number of 
schools that were selected from each district by treatment Is shown In Table 
3-1. 

Recruitment of Teachers Within Schools 

The schools In the sample varied according to the number of teachers 
assigned to grades four and five. Of the 15 sample schools, eight had two 
fourth and two fifth grade teachers; four had three or more teachers at a 
grade; and two had a single teacher at each grade. 

To make the sample of teachers within each school as similar as 
possible, two teachers were selected from each grade In those schools 
containing two or more fourth and fifth grade classrooms. The principal In 
the principal Involvement treatment In District 2 requested that all three 
fifth grade teachers In his building participate In the study. This request 
was honored. One District 3 teacher In the regular Inservlce group went on 
maternity leave shortly after receiving training and was eliminated from the 
sample. The number of teachers by c;rade per school Is Indicated In Table 
3-2. A detailed presentation of the district, school, grade level, sex, and 
treatment group of each participating teacher Is shown In Appendix A, 

Recruitment of Principals Within Districts 

A total of five principals from the three districts were selected to 
serve In the principal Involvement treatment. Three principals from the 
large district participated In this treatment condition; one principal from 
each of the smaller districts participated. 

33 

27 



Table 3-1 

Treatment Group Composition 

Schools Teachers 
N N 

Principal District 1 3 9 



In vol vement 
Group 


District 2 
District 3 




5 
4 




Regular 

Inservice 

Group 


District 1 
District 2 
District 3 


3 


10 
4 
3 




Control 
Group 


District 1 
District 2 
District 3 


3 


10 
4 
4 




Total 




15 Schools 


53 Teachers 
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Table 3-2 



Grade Level and Sex Composition of Teachers In Treatment Groups 




Principal 


Regular 






Invol vement 


Inservtce 


Control 




Group 


Group 


Group 




N 


N 


N 


Grade 4 


8 


8 


10 


5 


1Q 


9 


8 


Sex N 


7 


8 


7 


F 


n 


9 


11 
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§ Tht siiM proctduft for rtcrulting principals was ustd In a11 thrat ^ 

districts. Project staff presented an overview of the study, Including a 
general description of the principal Involvement treatment, to central office 
staff and building principals In each district during December 1982. The 
administrators were told that selection of principals for participation In 
the principal Involvement treatment would be left to the discretion of each 
district. Project staff Indicated that It would be desirable from a research 
standpoint to choose principals on a random basis, but the use of random 
selection procedures was not a condition for the district's participation In 
the study. 

As It turned out, none of the three districts randomly assigned 
principals to the principal Involvement treatment. Rather, principals were 
selected on the basis of their Interest In the study, their reputation as 
effective leaders, or on their current work load. Work load was a factor 
because some principals had been assigned leadership roles In other projects 
and had no time available for the leadership role called for In the present 
study. 

According to administrators In each district, prlncljuls were assigned 
to the regular Inservlce or control treatment based primarily on their 
expressed Interest In the experiment. Principals indicating a desire to 
learn about the Good and Grouws model were assigned to the regular Inservlce 
group. Principals who did not express a particular Interest In the model 
were assigned to the control group. 

Financial Incentives for Teachers and Principals 

Each of the ten schools assigned to the control or regular Inservlce 
treatments received $100 as an Incentive for the participating teachers. The 
control group teachers were viewed as "participants" because even though they 
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did not «tttnd training ttsslons, thtir participation was rtqulred for tha 

research observations and testing. 

Each school assigned to the principal Involvement treatment received 
$2S0 as an Incentive. Schools assigned to this treatment received more money 
In recognition of the extra demands that the treatment placed on principals. 
The principals were asked to attend training sessions, to make two formal 
evaluations of each participating teacher's lessons, and to complete several 
other tasks. 

The money was paid to a general fund In each of the 15 participating 
schools, to be used In accordance with school priorities and policies. 

Description of Sample 

Description of Districts 

All three districts were located In Oregon's mid-Willamette Valley. 
District 1 was situated In a large, suburban community and served about 7,000 
students. The other two districts, separated by only about ten miles, served 
small communities. Each had a student population of approximately 2,400. 
These small districts had been engaged In collaborative school Improvement 
projects for over ten years and were both using the same elementary 
mathematics program. 

Description of School s 

Information was collected on the size of schools In the sample, the 
grade levels they served, and the policies governing student classroom 
assignment for mathematics Instruction at grades 4 and 5. 

The student population of the elementary schools In District 1 ranged 
from 147 to 740. Each of these nine schools had classrooms from kindergarten 
or grade one through grade six. 
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Tht stu^tnt population of schools In tht SMipU from tht snail districts ^1 
ranged from 254 to 465. Five of the six schools In these districts had 
classrooms at grade one through six* One school Included only grades four 
through six* 

The dominant policy guiding classroom placement of children for 
mathematics Instruction across all three districts was to maintain 
heterogeneous classrooms, that Is, classrooms containing children with widely 
different backgrounds, abilities, and learning characteristics. However, 
three schools In District 1 and one school In the small districts practiced 
w1 thin-grade ability grouping for mathematics Instruction. For example, one 
fourth grade teacher might teach math to all high-performing students at that 
grade level, while the other provided Instruction to all moderate-to-low- 
performlng students. 

Teachers' Inservlce Background 

The project staff collected Information on district-sponsored Inservlce 
programs for teachers offered during the past several years. Virtually all 
teachers In District 1 had taken at least one course In I TIP conducted by the 
director of staff development for the district. The director Is a highly 
respected ITIP trainer, and she conducted the training sessions In District 1 
In the present experiment. 

Teachers In the small districts did not share this background of 
experience, nor for that matter any common background of Inservlce work. 
However, they did have a common experience In Valley Education Consortium 
(VEC) mathematics program (described later), and In working with building 
colleagues to Identify and resolve problems encountered In Implementing this 
program. 
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Description of Principals 

All of the 15 principals of the sample schools were Interviewed prior to 
the Intervention (January 1983) to obtain background Information on school 
policies and procedures and on various Instruction-related practices 
generally followed In their school. A copy of the Interview schedule that 
was used In these meetings Is In Appendix B. 

Project staff wished to determine the extent to which the practices 
called for In the principal Involvement treatment were consistent with 
principal's pretreatment practices, or differed from them. This treatment 
called upon principals to attend the training sessions for teachers on the 
Good and Grouws model and to observe teachers as they worked with the model 
In their classrooms. The principals were to engage In several other 
practices, but participation In training and observation of classroom 
practice represented the heart of the treatment. 

Several Important findings emerged from these Interviews. Perhaps the 
most Important was that principals assigned to the principal Involvement 
groups and the regular Inservlce groups had participated In Inservlce 
training sessions for teachers more often than had principals In the control 
group. This was the same In all three districts. Virtually all principals 
assigned to the treatment groups Involving training Indicated that they tried 
to participate In all^ Inservlce programs offered In their buildings. These 
Included Inservlce programs sponsored by the publisher of a recently adopted 
textbook series, workshops on computer use led by a member of the school 
staff, and a presentation of new techniques In classroom management. By 
contrast, principals In the control schools made conments like: "I don't 
have much Involvement In teachers' Inservlce," or "I participate very little 
in teachers' Inservlce." Principals In control schools generated fewer 
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txMplts Of Instrvlct training stssions th«y had attandtd than did principals 
In treatment schools. 

A second finding was that District 1 principals In the schools assigned 
to receive training were more likely to use ITIP as the primary basis for 
Instructional supervision than were principals In control schools In that 
district. Principals In the small districts, regardless of the experimental 
group to which they were assigned, did not use any particular model of 
Instructional effectiveness as a guide for supervision. Information on kay 
principal practices Is summarized In Table 3-3. 

Description of District Mathematics Programs 

The elementary mathematics program In District 1 contains general 
learning goals for each grade level, specific 1 tructlonal objectives, and 
Illustrative teaching and assessment procedures that relate to the stated 
goals and objectives. This Information Is published In a "Curriculum Guide 
Notebook" for teachers. The district's textbook series In matuematlcs 
appears to correspond to a large degree with the published curriculum. * 

The elementary mathematics program In the two small districts was 
developed through the Valley Education Consortium (VEC), an organization of 
small school districts and educational agencies In a three county area In 
Western Oregon. The major purpose of VEC Is to foster long-term school 
Improvement. (A description of the Consortium and Its work < presented In 
Appendix C.) Staff from Districts 2 and 3 had extensive Involvement In the 
development of VEC's mathematics program. The elements of this program are 
listed In Table 3-4. It was developed on the assumption that a district 
would adopt a textbook series and obtain related materials that teachers need 
to promote students' achievement of designated learning goals. Informal 
discussions with teachers In the two districts implementing the program 
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Table 3-3 



Key Principal Practices by District and Treatment Prior to the Experiment 



Principal Involvement 
and 

Teacher Inservlce 

Groups Control Groups 



District 1 
Clarge district} 



Districts 2 & 3 



frequent participation In 
Inservlce training for 
teachers 

extensive use of the I TIP 
model In the supervision 
process 

frequent participation In 
Inservlce training for 
teachers 

supervision not guided 
by any particular 
Instructional model 



Infrequent participation 
In Inservlce training for 
teachers 

more limited use of the 
I TIP model In the 
supervision process 

Infrequent participation 
in Inservlce training for 
teachers 

supervision not guided by 
any particular 
Instructional model 
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Table 3-4 

Major Elements of the 
VEC Mathematics Program 
Grade 1-8 

learning goals organized by grade level 

test Item pods and other assessment procedures that teachers and 
administrators can use to monitor student progress toward goal attainment 

computer programs for scoring, analyzing, and reporting test Information 

grade-level handbooks for teachers that provide guidelines for goal -based 
planning, assessment, and decision-making 

guidelines for assessing and reporting students* progress toward graduation 
competence requirements 

goal-based program evaluation and reporting procedures 

guidelines for choosing and using pro gram- related standardized tests 
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Irullcattd th«t tht ttxts In ust rtlatt rusonably wtll to tht YEC curriculum, 
although several teachers suggested that supplementary material Is needed to 
accomnodate the full range of specified learning goals. 

The main difference between the VEC program used by the two small 
districts and the program In place In the large district lies In the area of 
assessment. The VEC program contains extensive test Item pools that teachers 
can use on a day-by-day basis to assess student progress toward goal 
attainment. Administrators are able to use the same test Item pool and to 
monitor program effectiveness through district-managed mid-year and 
end-of-year tests. 

The textbook series adopted by the large district contains 
well -developed tests and assessment procedures, although these assessment 
resources are not as extensive as those available In the VEC program. 
Guidelines for using test Information also are less well developed. This was 
the case for both teachers and administrators. 

Training of Teachers In the Three Treatments 
Good and Grouws* Version of Training Program 

The training program used by Good and Grouws In their experiments was 
described by them as being a "treatment [that] consisted of two 90-m1nute 
training sessions and a 45-page manual that detailed the treatment and 
provided a base for teacher reference as necessary" (Good and Grouws 1979, p. 
356). In the first session teachers were given a brief orientation to the 
experiment, an explanation of the program, and a 45-page manual (reproduced 
In Good and Grouws 1981) to read after the session. Two weeks later a second 
session was held "...to respond to questions that teachers had about the 
meaning of certain teaching behaviors and to react to any difficulties that 
the teachers might have encountered" (Good and Grouws 1979, p. 35£). 




ProftssQr Qrouws ntt with the research staff to discuss the training 
sessions so that the^ could be repeated In the present experiment. A manual 
describing the training sessions was not available, so It was helpful to meet 
with Professor Grouws, who had conducted the training sessions In several of 
the experiments Involving the Good and Grouws' program. 

Everyone agreed In these meetings that It would be advantageous to 
double each session from ninety minutes to three hours In order to strengthen 
the training. For example, the additional time made It possible to show 
teachers two videotapes Illustrating Good and Grouws' Instructional model. 
These videotapes were developed subsequent to Good and Grouws' experiments. 
Also, the extended three-hour sessions gave the trainer additional time to 
address teacher questions and concerns. 

The training program used In the present program Is described below. It 
should be noted that this training program was Identical for teachers In both 
the principal Involvement and regular Inservlce treatments. The one 
dimension on which the two treatments differed— namely, principal 
Involvement— Is described In a subsequent section titled, "Principals' Role 
In Principal Involvement Treatment." Following that section Is a description 
of the control treatment. 

It Is Important to note that (a) teachers In the regular Inservlce 
treatment, (b) teachers In the principal Involvement treatment, and (c) 
principals of teachers In the principal Involvement treatment attended the 
training sessions together. Thus, regular Inservlce teachers had the 
opportunity to hear reactions from principals from other schools to Good and 
Grouws' model and to observe teachers interacting with principals about the 
model. Nevertheless, because the teachers and principals In the principal 
involvement group were from other schools in the district, the Influence of 
these participants on the regular inservlce teachers was at best indirect: 




tht principals htd no authority ovtr thmut and comnunl cation bttwten teachars 
from different schools Is Infrequent. Follow-up Interviews Indicated that 
there was no cross-school discussion of Good and Grouws' model for the 
duration of the experlnent, with the exception of the two training sessions. 

Orientation Session 

All teachers and principals In the three treatment groups attended a 
half-hour orientation session the week of January 10, 1983. A separate 
session was held In each district. The following topics were covered: 
purpose and design of the experlnent; general nature of the training program; 
data collection procedures; procedures for preserving data confidentiality; 
the need to avoid treatment contamination; scheduling; and project staff. 

Training Sessions 

The training was provided separately for each school district, and an 
Indigenous trainer was employed In each district to conduct that district's 
two sessions. This procedure was used so that participants In each session 
(teachers, principals, and trainer) could address the relationship between 
Good and Grouws' Instructional model and local school district conditions. 
It was thought that Implementation of the model would be facilitated If 
participants did not have to share "talk time" In the sessions with 
participants from other districts. 

The trainer In District 1 was the district director of staff 
development. She had worked as a teacher In the district and. In recent 
years, had been responsible for ITIP training (Madeline Hunter's "Elements of 
Effective Instruction" program referred to as "Instructional Theory In 
Practice") In the district. In District 2 the trainer was a district 
specialist In staff development and teacher supervision. The trainer In 




, District 3 was «n •Itntntary taachar who had bttn Involvtd In many Inservica 
activities with the district's teachers. A11 trainers had been with their 
districts for many years* 

The three district trainers were oriented to their role In the staff 
development program by Dr. Grouws during his visit. A member of the research 
staff provided additional training for them and gave them all necessary 
materials. In addition* he attended each training session In each district 
as an Informal technical resource to the trainer. On occasion he would field 
questions that were addressed to him by a participant or that the trainer 
wished him to answer. 

The first training session was held In January 31» 1983 (District 1), 
February 1 (District 2), and February 13 (District 3). The second training 
session was held about two weeks later on February 16 (District 2), February 
17 (District 1), and February 18 (District 3). Substitute teachers were paid 
by the research project so that teachers could attend the sessions during 
regular working hours. All participating teachers and principals were able 
to attend each session. 

The three district trainers were given an outline to follow. A copy of 
the outline Is presented In Appendix D. It Is generally self-explanatory* 
with a few exceptions. The ASCD (Association for Supervision and Curriculum 
Development) videotape used In the first session was developed by Good and 
Grouws. The videotape Is titled Teaching Mathematics Effectively and Is sold 
by ASCD for use by teacher educators. It features Professor Good explaining 
the Instructional model, and also Includes brief segments of classroom 
teaching that Illustrate key elements of the model. 

The videotape used 1n the second session shows a complete math lesson 
(approximately 40 minutes) taught by an elementary teacher who uses each 
element of the model. The tape Is noi; narrated. Brle^ sepents of It were 
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ustd In tht ASCO vldtoUpt* It wa$ 1«nt to tht project by Proftssor Grouws* 

Whole group versus Individualized Instruction Is discussed on page 4 of 
the outline. This material was written for the trainer's use because some of 
the participating teachers emphasized Individualized Instruction, and It was 
anticipated that they might raise objections to the whole-group approach In 
Good and Grouws' model. 

The questions and answers on pages 8-9 were provided by Professor 
Grouws, who recalled typical questions from his training experience. He also 
provided the overhead transparencies mentioned on page 2 of the outline. The 
lesson plan form mentioned on page 2 and again on page 6 Is reproduced on 
page 10 of the outline. 

During the first session all participating teachers and principals were 
given a personal copy of the manual (approximately SO pages) used In Good and 
Grouws' fourth-grade experiment. 

The trainers generally followed the outline for the two sessions, 
although there were variations across districts. The trainer In District 1 
emphasized the relationship between the ITIP Instructional model and Good and 
Grouws' Instructional model. Trainers In Districts 2 and 3 related the model 
to local teaching conditions. The lesson plan form was used by only two 
trainers In just one of the sessions. 

The member of the research staff who attended the training sessions took 
notes on each session. A summary of his notes are presented In Appendix E. 
A CEPM staff member attended the second training session In District 2. A 
brief publlcatlor based on her Impressions Is Included In the same appendix. 

Debriefing Session 

Teachers and principals In the two trained groups In Districts 2 and 3 
voluntarily requested a third session at the conclusion of the experiment. 
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This stsslon M«s held Junt 10, 1983 In each district stparately with most 
teachers and both principals attending. The session was used by participants 
to share experiences and to discuss their current views of Good and Grouws' 
Instructional model and training format. A member of the research staff 
attended both sessions and took notes. A summary of these notes Is presented 
In Appendix £• 

Control Group Teachers 

Control group teachers and principals attended the orientation session 
described above. They were told that they were participating In an 
experiment on Improving mathematics Instruction. The control group teachers 
were asked to continue their regular math Instruction for the duration of the 
experiment. They were told that they could receive the Inservlce program 
used In the other treatments after the conclusion of the experiment. No 
Information about the Good and Grouws Instructional model was provided for 
them. 

The same observation and test data were collected from the control group 
as from the two trained groups. No feedback based on these data was provided 
to the control group, however, whereas the two trained groups did receive 
summarized results of the post and delayed observations and pre-achlevement 
test administration. 

Principal Involvement Treatment 

Design 

The principals In the principal Involvement treatment were expected to 
attend the training sessions for teachers on the Good and Grouws model, and 
to support teachers' use of the model In their daily mathematics instruction. 
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The primary form of support expected of principals was to make 
observations of teachers' math lessons (complete lessons were to be 
observed), and to provide feedback to teachers on their Instructional 
performance. Their observations were to take place Independently of those 
conducted by members of the research team. The principals were only expected 
to conduct observations of the fourth* and fifth-grade teachers who were 
participating in the experiment. 

In addition to observing teachers' classroom Instruction, the principals 
were expected to organize and moderate a meeting among participating teachers 
in their building to clarify and resolve problems encountered In using the 
model . 

The principals' responsibilities for Instructional leadership In the 
principal Involvement treatment, and the schedule for carrying them out are 
outlined In Table 3-5. 

Preparation and Support for Principals 

Shortly after the last Inservlce training session for teachers was 
completed (mid-February 1983), training sessions for principals In the 
principal Involvement group were held. The training sessions were Intended 
to clarify the nature of the tasks that principals were to perform and to 
offer the support and training that principals needed to complete these 
tasks. 

One training session was held for the two principals from the small 
districts, and one was organized for the three principals from the larger 
district. Project staff and the district trainers served as facilitators of 
these sessions. 

The sessions had several specific purposes. One was to discuss with 
the principals the responsibilities outlined on the task-timeline sheet 

49 43 



Table 3-5 

Leadership Responsibilities of Principals 
In the Principal Involvement Treatment 



Task 

Principals were expected to: 



Timeline 



1, Attend the two-and-one-hal f-day 
training sessions for teachers; 



by February 17 



1 



Observe one complete math lesson 
for each teacher In their buildings 
who received training (to Include 
pre- and post-conferences); 



by April 1 



3. Arrange and facilitate a 30 to 

40 minute meeting with participating 
teachers In their buildings to discuss 
and resolve any problems tt^at may have 
developed around using the model. 



by April 15 



4. Make another observation of each 
participating teacher's math lesson 
Including pre- and post-conferences. 



by May 15 
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(T«b1« 3-5)* Also, tht princip«1s wtrt givtn an opportun1t;y to review and 
refine the tasks In light of their particular clrcunstances. As It turned 
out, all principals Indicated that they were comfortable with the tasks and 
timelines outlined. 

Another purpose of the training sessions was to Introduce principals to 
different options for observing teachers' math lessons. Prior to the 
sessions, project staff and district trainers had Identified three different 
procedures principals might use for conducting observations: (1) using a 
checklist that identified the key Instructional practices called for In Good 
and Grouws' model (Appendix D); iZ) using the more detailed coding system 
that memters of the research team were employing for their observations; or 
(3) making a verbatim record of teachers' statements. The verbatim record 
was Included as an option because the principals from the large district had 
received training In recording verbatim teacher statements and were 
accustomed to using this approach. 

Each of these options was discussed with the principals. No one 
approach was presented as preferable to the others. It was Intended that 
each principal would choose the procedute that he or she found most 
appropriate and- workable. 

The training session In the large district also focused for a brief 
period of time on the format used to report results on the 
criterion-referenced pretest that was administered In January as part of the 
study. Administrators In the large district were unfamiliar with this 
format, so time was set aside to answer questions the principals had about 
It. 

The principal Involvement treatment did not require principals to make 
any particular use of the pretest results. The reporting format was 
discussed with principals In the event that teachers In their building, who 
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received pretest results for students In their c1 asses » had questions about 
the format. 

At the end of April, after principals had had an opportunity to observe 
teachers' math lessons and to discuss with teachers the Implementation of the 
Good and Grouws model, a second meeting of principals was convened. As with 
the earlier training session, one meeting was organized for the three 
principals from the large district, and one for the two principals from the 
small districts. 

These meetings were Informal and lasted about 40 minutes. ^ Their purpose 
was to provide an opportunity for principals to report progress and share 
concerns about their roles as Instructional leaders. It was thought that 
principals might learn from each other's experience In the project. The 
meetings also provided a chance. for project staff to check whether the tasks 
assigned to the principals were being carried out. 

Implementation of the Treatment 

As stated above, principals In the principal Involvement treatment had 
three responsibilities: (a) to attend the two training sessions for 
teachers; (2) to make two observations of each participating teacher's math 
lessons. Including pre- and post>observat1on conferences; and (3) to 
facilitate meetings among participating teachers concerning Issues 
encountered In using the Good and Grouws model. Information Is provided In 
this section on the degree to which these responsibilities were fulfilled. 

Attendance at Inservlce training sessions. All principals assigned to 
the principal involvement treatment attended teacher training sessions. 
Principals were not given any special Instructions for participating In these 
sessions, though in one district the principal made several strong comments 
about the Importance of using various practices identified in the model. For 



46 



V txanpUt he strtsstd the need to essign homework In accordance with the 

policies outlined In the model even though several teachers expressed 
reservations about students' willingness or capacity to complete homework 
assignments on a regular basis. 

Conducting observations of teachers' math lessons. Information on 
frequency of observations was obtained through Interviews with the principals 
In late May and early June 1983. 

Two of the principals In District 1 recorded teacher statements verbatim 
as this was the approach to which they were accustomed. One of the 
principals stated that "It was helpful to have options to choose from" In 
developing an observational approach, but found the clinical supervision 
practices she generally used were applicable to the present project. Her 
approach was to "Identify something that teachers do well and something they 
can Improve." According to this principal, the behaviors called for In the 
Good and Grouws model were essentially the same as those called for In 
"Instructional Theory Into Practice," so little adjustment was needed to 
carry out the supervision process, other than learning a new set of labels 
for Instructional behaviors. 

The third principal In District 1 and the two principals In District 2 
and 3 used the checklist developed by project staff for structuring classroom 
observations. Each found the checklist to be a simple method for focusing 
and recording observations. 

One principal In District I and one In District 2 made the two 
observations of each participating teachers' math lessons that were specified 
In the treatment. One principal In the large district made three 
observations of each participating teacher. Another principal In this 
district made two observations of three of the trained teachers, but only one 
of the fourth. One principal In the small districts made only a single 
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observation of each teacher. The number of observations completed by each 
principal Is summarized In Table 3-6. 

Conferences. All principals In the principal Involvement treatment held 
brief pre-observatlon conferences with teachers to prepare for each classroom 
observation. The preconferences were arranged In order to establish a 
specific time to carry out the observations and to clarify or confirm the 
observational procedures. to be used. 

Each principal also held post-observation conferences with teachers. 
These conferences were used to share with teachers Information collected 
through the observations and to discuss any problems that may have arisen In 
Implementing the model. 

All participants reported that teachers were Implementing most of the 
practices Identified In the active teaching model » and the post-observation 
conferences served to reinforce these practices. One principal submitted to 
project staff the checklists he had used to guide and record observations of 
three different classrooms, though these checklists had not been requested. 

The checklists showed that the clear majority of desired practices were 
being followed In each classroom. The records Indicated, however, that one 
teacher had not conducted mental computation exercises, and another teacher 
had spent less than 20 minutes on the development phase of the lesson. The 
principal In this school said that he had discussed these omissions and 
deviations from the model with the teachers and came to the conclusion that 
there were valid reasons for not doing mental computation during every lesson 
or for adhering precisely to the recommended time allocations. "Kids get 
tired of mental computation If they have to do It each day," a teacher had 
told him. Another teacher Indicated that she needed to spend more time on 
review that the recommended eight minutes when students' homework revealed 
that they had problems with a previous lesson. When review was lengthened, 
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Table 3-6 

Frequency of Observations Conducted by 
Principals In the Principal Involvement Treatment 



Principals 

District 1 

Principal A 
B 



Observations 



2 observations per teacher 

2 observations of 3 teachers; a 
single observation of the fourth 
teacher 

3 observations per teacher 



Districts 2 and 3 

Inclpal 0 
E 



2 observations per teacher 
1 observation per teacher 



ERIC 



55 

49 



developiMnt was shorttn«d, she explained. 

A principal In District 1 used conferences to "problem solve" with two 
teachers concerning ways of adapting the model to heterogeneous classes, like 
preparing a separate development section for each of two or three groups 
rather than making one presentation to the class as a whole. This 
necessitated shortening the development section, the principal noted. But In 
her view the benefits of tailoring the presentation of new material to 
students' learning backgrounds and abilities In some cases outweighed the 
costs of shortened development sections. 

A principal in District Z highlighted In conferences the Importance of 
"manlpulatlves" In the development phase of the lesson. He observed that 
teachers generally did not use manlpulatlves. He concluded from the 
conferences that he needed to take steps to ossure that high-quality 
manlpulatlves were more readily available to ''eachers. 

Facilitatin g teacher use of the model. None of the principals convened 
a special meeting of teachers to discuss the use of the Good and Grouws 
model. In all schools In this -treatment, however, teachers met among 
themselves Informally or during regular staff meetings to exchange views 
about the mode). In no case was this Interaction explicitly orchestrated by 
the principal. 

The failure of principals to organize a separate meeting for teachers In 
April ref1ecte<'l the principals' considered judgment rather than an 
unwillingness to comply with treatment specifications. All principals In the 
treatment had conversed Informally with teachers about the model, had 
observed formally at least one math lesson taught by each teacher, and were 
well aware that teachers had talked about the model among themselves and 
shared techniques for adapting the model to their classrooms. It seemed 
unnecessary to organize a separate meeting In April to discuss matters that 
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«1rt«4y hid bttn discusstd. 



Principals' Reactions to their Participation In Experiment 

Reactions to the wodtl . Principals In the principal involvement 
treatment had {^snerally positive reactions to the model. Representative 
comments made by principals In the interviews conducted In late May and early 
June follow: 
District I 

Principal A: "The model was complete and well structured. It was 
easily related to ITIP." 

Principal B: "This active teaching approach seems to be an effective 
way of teaching, not Just math, but in other areas, too. Our new 
reading series includes more teacher-directed activities than in 
the past. It makes sense to have kids spending a large amount of 
time Interacting with the teacher.... Also, the 15-m1nute limit 
on homework was helpful, along with the notion that homework 
should be regularly assigned." 
Districts 2 and 3 

Principal C: "I liked this method. It helped teachers organize 
lessons better. There were clear checkpoints to make sure things 
were going as they should." 
Principal D: "This is a solid approach. It helps teachers keep 
students on task. It helps them to be more consistent and 
organized in their teaching..." 
One principal raised doubts about certain features of the model. He saw 
value in the overall structure of the model, and reported that the 
recommendations on homework appeared to be well received by both teachers and 
students. He was concerned, however, about what he perceived to be the 

5V 

51 



0V2r1y prescriptive time allocations for various phases of a lesson and the 
"bias" toward whole-group Instruction embedded In the model. He also 
criticized the training sessions for falling to discuss the theoretical 
underpinnings of the model. 

All the principals In the principal Involvement treatment commented on 
the difficulty that some teachers had In using the model with heterogeneous 
classes. One principal Indicated that the project "stimulated our thinking 
about ability grouping." Another principal suggested that teachers who had 
practiced some form of Individualized math Instruction for the first semester 
were hard pressed to alter their practices In February simply on the basis of 
two brief training sessions on the Good and Grouws model. This principal 
recomnended that the model should be Introduced at the beginning of the year 
If It Is to be Introduced at all. 

Reactions to their role as Instructional leaders. All of the principals 
Indicated that their attendance at the training sessions provided a common 
basis for subsequent observations and discussions with teachers. Since they 
regularly attended Instruction- related Inservlce activities, participation In 
the training on the Good and Grouws model was not perceived as burdensome or 
unusual In ar^ way. 

One of the principals, however, perceived that his district was 
uncertain regarding teachers' Implementation of the model. He wondered 
whether he was supposed to require teachers to use the model, even If they 
had misgivings about It, or whether teachers were free to use, adapt, or 
reject the model as they saw fit. He would have preferred the district to 
convey a clear mess'^e on this Issue at the training sessions. 

A second principal In the large district echoed this concern about the 
districts' Intentions. He stated that he "would have liked more explicit 
direction from the district." 
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with respect to conducting cUssroom observations, all the principals 
Indicated that the treatment called for more frequent formal observations of 
math Instruction than they ordinarily would carry out In a comparable time 
period. None of the principals, however, suggested that the observations 
called for were an unreasonable burden. The principals found the 
observations to ^e of value In assessing the strengths and limitations of the 
model and In focusing teachers' attention on specific elements of the model 
that they were using or not using. 

Principals' Roles In Other Treatments 

Regular Inservlce treatment. Principals In the regular Inservlce 
treatment were requested In the orientation sessions (described earlier) not 
to participate in the Inservlce training sessions and not to attempt to 
observe or evaluate teachers' use of the Good and Grouws model. They were 
asked to permit project staff to collect data In their building, but were not 
expected to participate In the stu4y In any other way* 

To verify that principals In the regular Inservlce treatment refrained 
from Involvement In the stu4y. Interviews with each of these principals were 
conducted In May, 1983. The Interviews Indicated that none of the principals 
had attended the training sessions, and none had asked teachers about the 
Good and Grouws model, or had tried to observe teachers' use of the model. 
Several of the principals in this treatment stated that they had heard 
Informally from participating teachers that It was difficult to use the Good 
and Grouws approach In heterogeneous classrooms. These same principals also 
indicated that they had learned through Informal comments that the model 
called for regular homework, a practice the principals approved. However, 
none of the principals In the regular Inservlce treatment took any steps to 
enhance teachers' use of the active teaching approach. 

53 



One principal In District 1 who was assigned to the regular Inservlce 
treatment also taught fifth grade mathematics. This principal worked In a 
small school In which the building administrator customarily assumed some 
teaching duties. The principal participated In the Inservlce training 
sessions, as did the principals In the principal Involvement treatment, but 
he participated In the role of a teacher, not an "Instructional leader," and 
did not make formal observations of other teachers' mathematics lessons. 

Control treatment. Interviews also were conducted In late May, 1983 
with principals In the control groups. None of these principals reported 
having any Information about the Good and Grouws model, or about the tasks 
that principals In the principal Involvement treatment were expected to 
perform* None Indicated that they had made any attempt to observe teachers' 
use of the model. 

Observational Measures of Classroom Instruction 

A classroom observation Instrument was developed and used In the 
experiment. One purpose of the measure was to determine whether teachers who 
received training In the experiment made changes In their Instructional 
behavior In conformance with Good and Grouws' active teaching model. A 
related purpose was to determine whether students of the trained teachers 
Increased their on-task behavior during math Instruction. Together, the 
observation Instrument and the student achievement tests described below 
provide a set of dependent variables that were hypothesized to be affected by 
the principal Involvement treatment and the regular Inservlce treatment. 

Another use of the observation Instrument was to measure fidelity of 
treatment Implementation. The major outcome of the experiment was gain In 
student mathematics achievement. If gains were not observed, a por»s1ble 
explanation would be that teachers did not Implement the Instructional model 
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In the two treatments that Involved training (principal Involvement and 
regular Inservlce). Lack of fidelity of treatment Implementation Is a 
persistent problem In field experiments In education (Charters and Jones 
1973). The classroom observation Instrument provides a direct measure of 
fidelity of treatment Implementation In this experiment. 

As stated in the section on experimental design, this Instrument was 
used to observe teachers' math Instruction at three points In time. It was 
used shortly before the training period, again about three weeks after 
training, and finally about ten weeks after training. 

The Observation Instrument 

Initial consideration was given to using or adapting the checklist 
format developed b^ Good and Grouws for assessing Instructional effects of 
their training program. An example of their checklist format Is shown In 
Appendix F. The type of data yielded by this Instrunent Is shown In Table 
2-4, which presents classroom Instruction effects observed by Good and Grouws 
In their fourth-grade study. 

The decision was made to retain most of the classroom variables In Table 
2-4 but to measure them more precisely. Instead of relying on time 
estimates, observers recorded the exact amount of time that each 
observational behavior occurred. 

The observation Instrument used In the experiment Is shown In Appendix 
G. The first page of the Instrument Is used to record Identifying 
Information and answers to six questions about the planned lesson. These 
questions are addressed to the teacher before the lesson begins to help the 
observer become oriented to the lesson and to help the observer decide which 
observation categories to assign to behaviors occurring during the lesson. 

The next four pages make up the main part of the Instrument and are 
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Identical to each other with one exception. The design of this part of the 
Instrument can be understood by examining the top half of page two. The 
first row contains thirty-second Intervals. The next 13 rows contain 
observation variables. With the exception of the row labeled "No. of 
students off-taslc," all of the other rows refers to observation of teacher 
behavior. Each of these rows refers to a single observation variable with 
the exception of the row labeled "Seatwork (monitored +)." This row Is used 
to record two observation variables: monitored seatwork (+) and unmonltored 
seatwork (-). 

In using the Instrument, observers use a stopwatch to determine 
30-second Intervals. At the end of each Interval, the observer circles the 
behavior that occurred during that Interval. A 30-second Interval was used 
because pilot work Indicated that teachers seldom engage In more than one of 
the behavior categories during that period of time. If a teacher should use 
two of the behavior categories within a 30-second Interval, the observer 
would circle both categories. Each category would be assigned a value of 15 
seconds for the purpose of this data analysis. 

Another feature of the Instrument Is that observers switch from 
recording teacher behavior to recording student off task behavior after every 
five minutes of the lesson. At these points In time the observer Is given 30 
seconds In which to look at each student In the class and determine whether 
he/she Is on or off task. The observer counts the number of off task 
students and enters this total In the row labeled "No. of students off task." 
The "5:00" column In the top half of page 2 contains only a line on which to 
record the off-task student count. This feature, which Is repeated at 
five-minute Intervals, Is designed to cue observers to switch from recording 
teacher behavior to recording student behavior. 

The bottom set of rows on page 2 and the set of rows on pages 3-5 are 
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used for continuous recording of behavior as the lesson progresses. The 
Instrument contains space for recording lessons up to 60 minutes In length. 
The Instrument can be modified easily by adding pages to record lessons that 
last beyond 60 minutes. 

A manual was developed to assist observers In recording behavior ; . the 
Instrument. A copy of the manual Is presented In Appendix H. The manual 
contains a definition of each behavior and a set of decision rules to help 
observers record the behaviors reliably. These rules were -developed based on 
experience during observer training and Initial data collection* 

The last page of the Instrument (Appendix G) Is used for high Inference 
ratings of student comprehension (questions 1 and 2), and a rating of verbal 
problem-solving occurrence (question 3). Item 4 Is used by observers to 
record Information about classroom organization and unusual occurrences. 

Content validity was the major validity concern In designing this 
Instrument. The observation variables and manual needed to be Inclusive of, 
and defined In the same way as, the Instructional behaviors presented In Good 
and Grouws' Teachers Manual (Good, Grouws, and Ebneler 1983) used In the 
present experiment. Dr. Grouws made a visit to Oregon to assist the research 
team In meeting this goal. Me reviewed the philosophy of the Instructional 
model, and shewed several videotapes of teachers using the model. While 
showing the videotapes, he and the research team discussed the meanings of 
each Instructional behavior and distinctions between similar-appearing 
behaviors. He also discussed his experiences with classroom observation 
using his own instruments. 

In addition to this meeting with Dr. Grouws, the research team 
continually consulted Good and Grouws' Teachers Manual to be certain that the 
definitions In the observation manual (Appendix H) were consistent with the 
definitions In the Teachers Manual. Also, the summary of the Instructional 
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model (see Table 2-2) was consulted to check that the observation Instrument 
Included a11 of the essential elements of the model. The Instrument and 
observation manual were reviewed Independently by members of the research 
team until even'one was satisfied that they were content valid with respect 
to the Good and Grouws' Teachers Manual . 

* 

Data Collection 

Eight substitute teachers were employed by the research team to collect 
the observational data. Substitute teachers were selected because they were 
knowledgeable about classroom Instruction; because they were available at the 
times required; and because they were likely to be accepted by teachers as 
observers In their classrooms. All the observers possessed a teaching 
certificate, and all but one had experience teaching at the elementary school 
level. Also, all but one of the observers lived In or around Districts 2 and 
3.' The reason for this observer characteristic Is that observer training 
occurred In these two districts, and It was convenient to select observers 
fr<M the vicinity In order to minimize driving time. District 1 was 
approximately an hour's driving time from the main site used for observer 
training. 

All eight of the observers completed the training period satisfactorily 
and participated In the first round of data collection. Two of the observers 
were unable to participate In subsequent rounds because they obtained 
longer-term employment In the Interim. 

The observers received five half-days of training to learn how to use 
the observation Instrument reliably. Training Involved the scoring of 
videotaped math lessons and also scoring of "Hve" math lessons In control 
group classrooms In District 3. Observers also received three half-days of 
refresher training just prior to the second round of observation and again 
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just prior to the third round of observation . They were given very little 
Information about the nature of the experiment. They were never told the 
treatment group assignment of the teachers whom they observed. 

Classroom observations were scheduled at the teachers' convenience. The 
scheduling was arranged so that there would be no bias In assignment of 
particular observers to treatment groups or In time of week that a treatment 
group was observed. If a teacher was 111 or unavailable when an observation 
was scheduled, a make-up date was arranged. Observations were not scheduled 
on days when the teacher was giving a long test, when a student teacher was 
Instructing the class, or when an unusual event was scheduled. Also, no 
observations were scheduled on Mondays, because In Good and Grouws' 
Instructional model, weekly and monthly reviews are scheduled for that day 
(see Table 2-2). It was possible to observe the e1e.,:^nts of dally review, 
development, seatwork, and homework shown In Table 2-2 on the other four 
school days, except for Friday t when no homework was assigned; If an 
observation occurred on that day, the observation Instrument manual (Appendix 
H) specifies a procedure to be followed. 

Observer Reliability 

Twenty-five of the 159 observed lessons (53 teachers x 3«t1mes of 
observation) were scored by two observers. The 25 lessons were used to 
derive estimates of Inter-observer reliability. The lessons spanned the 
three districts, the three times of observation, and the three treatments. 
Also, many different pairs of observers from the set of eight observers are 
represented In the 25 lessons. 

Table 3-7 presents between-observer correlation coefficients for total 
time measures of the 13 observation categories, for lesson length, and for 
average number of off-task students. The coefficients are generally high. 
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Table 3-7 

Inter-observer Reliability for Observation Categories 
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ASS 1 gn noniQWurK 




Quiz 


.62 


Mental computation 


.'98 


Development 


.96 


Controlled practice 


.46 


Seatwork directions 


.85 


Monitored seatwork 


.73 


Unmonltored seatwork 


.84 


Check seatwork 


.81 


Lesson UngtK 


.99 


Off-task students 


.87 
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An exception Is the reliability coefficient for controlled practice (r-.46). 
This category occurred Infrequently In the sample of 25 lessons. Observers 
agreed perfectly on time spent In controlled practice In 21 of the lessons, 
and were within 2.5 minutes of each other In two or more of the lessons. In 
only two lessons was the disagreement excessive (0 minutes vs. 10.5 minutes, 
and 0 minutes vs. 7.5 minutes). 

Data Analysis Procedure 

The first step In scoring the data collected by the classroom 
observation Instrument was to compute the amount of time that each teacher 
spent using each of the 13 behavior categories. When two observers had 
collected data on the same teachers at the same point In time, the observers' 
time totals for each behavior category were averaged to yield a single time 
score. 

The off- task variable was derived by first summing the number of 
off- task students at each five-minute Interval In the lesson. Next the sum 
was divided by the number of such Intervals to yield a score that represented 
the average nunber of off-task students for that lesson. When two observers 
had collected data on the same class, their scores for this variable were 
averaged to yield a single score. 

The teachers' time scores for the 13 behavior categories were analyzed 
in three ways. The primary statistical analysis was to determine whether the 
teacher's time use for a behavior category met the criterion established by 
the researchers for that category. The reason for using a time criterion is 
that Good and Grouws' Instructional model specifies minimal, maximal, or 
optimal amounts of time that particular behaviors should be used. Therefore, 
It seemed Important to determine the percentage of teachers In each treatment 
group who used each behavior category for the criterion time period. The 

67*' 



ratlonaU for the time criterion Is provided In Appendix I. 

The second statistical procedure Involved computing the percentage of 
teachers In a treatment group who used a behavior category for any period of 
time. The third procedure was to compute the actual amount of time that 
teachers In each treatment group spent on each behavior category. These two 
analyses of the observational data provide a fuller description of teachers' 
Instructional behavior than would be provided by just computing percentage of 
teachers meeting a time criterion. 

Measures of Students' Mathematics Achievement 
Both curriculum-referenced tests and norm-referenced tests were used as 
pre- and post-treatment measures of student achievement In mathematics. 
Tests developed by the Valley Education Consortium (VEC) for Its 
Instructional program In mathematics served as the curriculum-referenced 
measures. Various levels of the California Achievement Test, published by 
McGraw-Hill, served as the norm-referenced measures. 



The VEC Tests 

The VEC grade 4 and grade 5 mid-year tests In mathematics were 
administered In January 1983 In each of the three districts tc fourth- and 
fifth-grade students, respectively. These tests were a pre-treatment measure 
of student achievement. The VEC end-of-year tests In mathematics for grades 
4 and 5 were administered In May 1983 In each district as a post-treatment 
measure of achievement. 

The VEC tests were prepared from test item pools Included In the 
Consortium's mathematics program. The Item pools had been developed by 
teachers, working collaboratively with staff from an educational research and 
development agency. 
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The YEC tests were designed to assess the knowledge and skills 
Identified In the VEC mathematics program, which the two small districts In 
the study were Implementing. Teachers In the two districts critically 
reviewed the tests and determined that the final selection of test Items 
matched designated learning goals In the VEC mathematics program. A oanel of 
teachers and aoimnlstrators In the large district participating In the stu((y 
analyzed the relationship between the district's curriculum and the VEC 
curriculum. They found a close match between the two, and so It seemed 
reasonable that the VEC tests would have curriculum validity In the large 
district as well as In the two small districts. In fact, one of the reasons 
administrators In the large district wished to participate In the study was 
to gain access to and familiarity with the VEC testing program. 

Differences between grade A and grade B tests. The VEC tests at grades 
4 and 5 have different Items, reflecting the differences In learning outcomes 
that students are expected to achieve In these grades. For example, students 
In grade 5 are expected to be able to work with decimals and the radius of a 
circle, whereas students In grade 4 are not. 

Another difference between the fourth and fifth grade tests Is In the 
number of Items used to assess learning In various content areas. The number 
of Items by content area at each grade level Is shown In Table 3-8. Both 
mid-year and end-of-year VEC tests for a particular graoe level covered the 
same content areas. Included the same number of Items In each control area, 
and liad the same total niinber of Items. 

Differences between mid-year and end-of-year tests. Differences between 
the mid-year and end of-year tests at each grade level were slight. The 
tests covered the same content, but some of the 1tem-s za the end-of-year test 
were of better quality than comparable Items on the mid-year test. This was 
because teachers had had several opportunities to critically review the Items 




Table 3-8 

Number of VEC Test Items 
Content Area 



Content Area 
Numeration 
Whole Numbers 
Measureirent 
Geometry 
Fractions 
Decimals 
Total 



Number of Items 
5 

30 
10 
10 
10 

mm 
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on the mid-year tests after the results from them had been reported, and to 
refine Items that appeared weak. These changes, however, did not Influence 
In any appreciable wi^ the difficulty level of the Items. 

The CAT 

The mathematics section of the California Achievement Test (CAT) was 
used as an additional measure of student achievement. This test was used 
because each of the three districts participating In the stu4y administered 
the CAT on an annual basis for the purpose of program evaluation. It 
appeared that data produced from the districts' testing programs would also 
have value In the research project. 

One complicating factor In using the CAT, however, was that the schedule 
of test administration In District 1 differed from the schedule followed In 
the small districts. In District 1, the CAT had been administered In October 
1982, and the district had not planned to administer It again dur'^ng the 
1982-83 school year. In the small districts, the CAT had been administered 
In May 1982, and the district planned on administering It In May 1983. 

Since the experiment required a post- treatment measure of achievement, 
administrators In District 1 agreed to administer the mathematics section of 
the CAT to fourth- and fifth-grade students In May 1983. This was In 
addition to the October 1982 administration. In the small districts, the r<lay 
1982 administration of the CAT served as the pre-treatment measure, and the 
May 1983 administration served as the posttest measure. Information 
regarding the use of the CAT tests In the experiment Is summarized In Table 
3-9. It should be noted that both District 1 and the two small districts 
administered the same levels of the CAT (14c for the fourth graders and 15c 
for fifth graders) for the posttest. District 1 and the small districts, 
however, administered different levels of the CAT and at different times for 
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Tatxle 3-9 
Administration of CAT Tests 

Grade 4 Students 



Date of Level of No. of 

Administration Administration Items 

Large Pretest October 1982 14c 85 

District Posttest May 1983 T?c SS" 

Small Pretest May 1982 13c 85 

District Posttest ftey"T553 Rc~^ 85 

Grade 5 Students 

Date of Level of No, of 

Administration Administration Items 

Large Pretest October 1982 15c 85 

District Posttest HayTsBZ T^c 55" 

Small Pretest May 1982 14c 85 

District Posttest May 1983 T3c 5& 
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the pretest. Since the fourth-grade students In the small districts were In 
the third-grade at the time of the pretest (May 1982), level 13c of the CAT 
was used. Since fourth-grade students were actually In the fourth grade at 
the time of the pretest (October 1982) » level 14c was used. The same 
differences occurred In the fifth-grade sample for the same reasons. 



Relationship Between the VEC and CAT Tests 

The VEC and CAT tests for grades 4 and S both cover the broad content 
areas of numeration, whole numbers, measurement, geometry, and fractions. 
The CAT tests for grade 4 and S and the VEC test for grade 5 also have Items 
on decimals. Decimals are not covered, however, on the VEC test for grade 4. 

Within broad content areas, the VEC and CAT test differ In two 
significant aspects. First, the difficulty level of many Items on the CAT Is 
higher than the difficulty level of VEC test Items keyed to similar learning 
objectives. For example, an Item on the CAT test from grade 4 requires 
students to subtract five-digit numbers, whereas the VEC test for grade 4 
only requires students to subtract numbers with four or fewer digits. This 
difference Is due to the fundamentally different nature of the two tests. 
Because the CAT Is a nonn-referenced test. It contains Items at varying 
levels of difficulty. The VEC tests are curriculum-referenced, however, and 
so they contain items only at the level of difficulty for the grade level 
being assessed. 

The second difference between the CAT and VEC tests Is the particular 
subsLills that are assessed. In the area of whole nun.i)ers, for example, the 
CAT 15c test includes an Item on prime numbers, whereas the VEC fifth grade 
test does not. On the other hand, the VEC tests assess measurement topics 
not c red in the CAT tests. 
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CHAPTER FOUR 
RESULTS 

This chapter presents statistical tests of the six hypotheses stated In 
Chapter !• The hypotheses specify effects of the three treatments (principal 
Involvement, regular Inservlce, and control) on the measured outcomes. 
The outcomes Involve teacher Instructional behavior, student mathematics achieve- 
ment, and student off-task behavior* 

The hypotheses were tested using a planned contrast procedure. A priori 
one-tailed t, tests were done comparing (a) the combined trained groups and the 
control group, and (b) the principal Involvement group and regular Inservlce 
group. Adjusted post and delayed treatment means were used In these com- 
parisons. The adjustments were made by analysis of covarlance, using the 
corresponding pre mean as the covarlata. The homogeneity of regression and 
homogeneity of variance assumptions of analysis of covarlance were checked 
to determine the soundness of the adjustment procedure. 

The analysis of covarlance was first done by an SPSS program to determine 
the regression effect, overall treatment effect, and adjusted means. A separate 
aralysis of covarlance was done, again using an SPSS program, to test the 
covarlate-by- treatment Interaction (that Is, to test the homogeneity of 
regression assumption). The homogeneity of variance assumption was checked by 
the Bartlett-Box test using an SPSS program. 

In several Instances where the assumptions of analysis of covarlance were 
not satisfied, a gain score analysis was done. The mean gain of each group was 
determined, and the difference between mean gains was tested using the t^ test 
for Independent means. This approach does not Involve the use of regression 
estimation. Also, although the t test Involves a homogeneity of variance 
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assumption. It Is not seriously affected by a violation of this assumption. 
The effects of violation are minimal when the N's are equal or nearly equal, as , 
In the contrast of the principal Involvement and regular Inservlce treatments; 
and they are not serious when the N ratio Is 2:1, as In the contrast of trained 
groups and the control group (Shavelson 1981, p. 423). 

Some researchers have criticized the validity and reliability of gain scores 
as a measure of change (e.g.. Linn and Sllnde 1977). However, a recent analy- 
sis of gain scores (also known as difference scores) found that the traditional 
criticisms are unjustified: "When only two waves of data are available, the 
difference score Is a natural and useful estimate of Individual change" (Rogosa, 
Brandt, and Zimoski 1982, p. 744). In fact, these researchers present evidence 
that gain scores are a more valid Indicator of change than the accepted method 
of residual 1 zed gain, which Is based on regression analysis. 

The null hypotheses were rejected at the .10 level (one-tailed test). This 
liberal alpha level was chosen to minimize the possibility of Type H errors, 
namely accepting null hypotheses as true when In fact they are false. 

The chapter concludes with several supplementary analyses Involving correla- 
tions between variables and a comparison of present results with those reported 
In Good and Grouws' fourt;h-grade experiment. 

Hypothesis 1. Teachers who participate In the staff development program (with 
or without principal Involvement) will vse active teaching techniques more 
frequently than teachers who do not participate In the program. 

Hypothesis 2. Teachers who participate In the staff development program with 
principal Involvement will use active teaching techniques more frequently 
than teachers who participate in the programs without principal Involvement, 
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The first step In testing these hypotheses was to determine the number of 
behaviors In Good and Grouws' instructional model that each teacher used for the 
criterion time period. Because 13 behaviors were measured, a teacher's score on 
this variable could vary between 0 and 13. The next step In the analysis was to 
compute descriptive statistics on this variable for each treatment group. The 
results of this analysis are shown In table 4-1. 

The three treatment groups were similar In their Instructional behavior at 
the time of the observed pre-treatment lesson. Teachers used an average of 
approximately five behaviors at the criterion level. Following the staff deve- 
lopment program, the two trained groups made similar changes In their behavior; 
each group used approximately eight of the behaviors at the criterion level. 
The control group remained at Its pre-treatment level. 

At the time of the delayed observation, the principal Involvement group con- 
tinued to make gains In Its use of the behaviors, whereas the regular Inservlce 
group regressed slightly. The control group continued to remain near Its 
pre-treatment level. 

Table 4-2 reports the tests of the two hypotheses at two times of obser- 
* vat Ion (post and delayed). Hypothesis 1 was supported: at both times of obser- 
vation the difference In number of techniques used to criterion level between 
the two trained groups and the control group was statistically significant. 
Hypothesis 2 was partially supported: the two trained groups were nearly iden- 
tical In the post-treatment lesson, but In the delayed lesson the superiority of 
the principal Involvement group approached statistical significance (p » .07). 

The homogeneity of variances assumption was not satisfied for the pre-lesson 
measure of number of techniques used to criterion level. However, the violation 
of the assumption does not appear serious since the adjustment of the post- 
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TABLE 4-1 

Number of Techniques Used to Criterion Level 
by Each Treatment Group 

Principal Regular 
Time of Involvement Group Inservlce Group Control Group 
Observation (N*18) (N«17) fN«18) 

Pre-Treatment M 4.94 5.29 5.17 

(SO) (1.59) (2.71) (1.69) 

Post-Treatment M 7.72 7.82 5.28 

(SD) (2.27) (2.51) (2.05) 

Delayed M 8.33 7.47 5.33 

(SD) (1.72) (2.37) (1.46) 
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MS 


p 
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D 


24.65 


5.14 


.03 


37.49 


7.82 


.001 


4.80 








.94 


NS 




2.94 


.05 




.32 
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TABLE 4-2 

Analysis of Covarlance Summary for 
Number of Techniques Used to Criterion Level 

POST TREATMENT MEANS 

Source df 

Regression (b ** .31) 1 
Treatment 2 

Within 49 

Homogeneity of regression 
Homogeneity of variances (pre) 
Homogeneity of variances (post) 
Adjusted means 

Principal involvement (PI) 7.79 

Regular inservice (RI) 7.77 

Control (C) 5.27 

PI and RI vs C t • 3.95 p • .001 
PR vs RI t • .03 p - NS 

DELAYED TREATMENT MEANS 

Regression (b « .26) 1 11.95 3.57 .07 

Treatment 2 43.84 13.08 .001 

Within 49 3.35 

Homogeneity of regression .10 NS . 

Homogeneity of variances (pre) 2.94 .05 

Homogeneity of variances (post) 2.05 NS 
Adjusted Means 

Principal Involvement (PI) 8.38 

Regular inservice (RI) 7.43 

Control (C) 5.53 

PI and RI vs C t » 4.86 p - .001 
PI vs RI t « 1.52 p « .07 
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treatment and delayed means resulting from the analysis of covarlance was very 
slight. 

The results presented above can be further understood by analyzing teachers' 
use of each of the 13 Instructional behaviors. The percentage of teachers In 
each treatment group who used each of the behaviors for the criterion time 
period Is shown In table 4-3. Substantial differences between the two trained 
groups and the control group were found on the behaviors most emphasized In the 
staff development program: assigning and checking homework; mental computation; 
and development and controlled practice. Fewer teachers In the trained groups^ 
however, were able to keep their transition behavior below two minutes. 

The slight superiority of the principal Involvement group on the delayed 
lesson (see table 4-1) Is described more specifically In table 4-3. 
Interestingly, the principal Involvement group and regular Inservlce groups are 
similar except for the seatwork phase of the delayed lesson. In that phase a 



directions, monitored seatwork, and checked seatwork for the criterion time 
level. 



tlonal behaviors Is shown In table 4-4. Substantial time differences between 
the trained groups and control group are evident on behaviors emphasized In the 
staff development program: assigning and checking homework; mental computation; 
development and controlled practice; and avoidance of unmonltored seatwork. The 
two trained groups differ from each other most In their use of developmental 
Instruction and seatwork. In the post-treatment lesson, the orlnclpal Involve- 
ment group spent less time than the regular Inserv^lce ^roup on developmental 
Instruction and substantially more time on unmonltored seatwork. In the delayed 



higher percentage of teachers In the principal Involvement 




The actual amount of time spent by each treatment group on the 13 Instruc- 
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Pwcaiugi of TNchiri In Gich TngMt Group 
Using Instructional Behaviors for Criterion Tine Periods 



Instnjctlonal Behaviors 



Transitions 

(2 nrinutes or less) 



nterruptlons 
no Interruptions) 



Review Previous VIork 
1 nrinute or more) 



Check Prior HoncMork 
at least 2 nrinutes) 



kssig\ Honework 
1 second or more) 



Quiz 

3 minutes or less) 



Mental Conputatlon 
at least 3 nrinutes) 



Oevelopnent 
at least 5 minutes) 



Controlled Practice 
at least 1 nrinute) 



Directions for Seatwork 
1 minute or less) 



Monitored SeatMork 
15 minutes or less) 



Uranonltored Seatwork 
2 minutes or less) 



Check SeatMork at End 
at least 1 nrinute) 



Percentage of Teachers within cui<ieijn5 



Principal involvnent 
Group (>W8) 



Pre 


61% 


rU5lt 


331 




331 


Pre 


50K 


Pnct- 




UCICQrCU 


sot 


Pre 


28X 




fill 






Pre 


ex 








fill 


Pre 


6% 


rUSt 






7P/L 


Pre 


67% 


rOSC 




ueiqyea 


POL 


Pre 


ex 


rOSC 




uei^yea 




Pre 


SOX 


rOSC 


Olh 


uei^yea 


cJJflt 


Pre 


33X 


TOSt 


OA 






Pre 


56% 


rOSt 


oust 


uei^yea 




Pre 


61% 


Post 


lOOX 


Delayed 


100% 


Pre 


56% 


Post 


67% 


Delayed 


94X 


Pre 


17% 


Post 


33% 


Delayed 


sex 



Regular inservlce 
Group (N»17) 




3^ 



0% 
59% 
65% 

18% 
69X 
71% 

76% 
78% 
82% 



Control Group 
(N"18) . 



33% 
67% 
22X 

11% 
OX 
11% 

22X 
6X 
6% 

5ex 

83% 
83X 



12%*^ * 

65?* 

47% 


22%V 
11% 
11% 


47%, ' 

82% 

76% 


44X 

39X. 
SOX 


47% 
53% 
41% 


33X 
11% 
6X 


65X 
53% 
47% 


67% 
44% 
72% 


41% 

8^ 
76% 


61% 
67% 
72% 


53% 
71% 
65% 


5ex 

50X 
67% 


24% 
41% 
29% 


17% 
28X 
17% 
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Length of Tire (In Minutes) for Each Instructional 
Behavior by Treatnent Group 



Instructional Behaviors 



I 


Principal Involvsneml 
Group (r^l8) 
M SD 


Regular Inservlo 
Group {f^n) 
M SO 


(N-18) 

M 


tM Ww|# 

SD 


Pre 


1.76 


1.45 


2.30 


1.89 


1.79 


1.47 


Post 


3.36 


2.27 


3.36 


1.91 


2.14 


1.72 


Delaiyec 


2.91 


2.17 


3.13 


1.95 


2.44 


2.91 


Pre 


.59 


.77 


.74 


1.48 


1 31 


2.00 


Post 


.84 


1.48 


.19 


.30 


.43 


.70 


Deliiyec 


52 


.82 


.51 


1.22 


.75 


1.28 


Pre 


1.73 


3.36 


1.82 


2.92 


1 63 


3.13 


Post 


3.54 


4.72 


2.66 


4.47 


6.81 


9.75 


Delayec 


2 03 


2.49 


1.24 


1.36 


1.00 


2.77 


Pre 


.25 


1.06 


.15 


.39 




1.86 


Post 


3.46 


3.53 


4.62 


3.78 


.00 


.00 


Delityec 


2.61 


2.47 


3.59 


4.01 


.74 


3.01 


Pre 


.03 


.12 


.12 


.28 


.16 


.33 


Post 


.51 


.43 


.84 


1.19 


.03 


.12 


Del6iye( 


7fl 


93 


.44 


.39 


.06 


.24 


Pre 


2.58 


3.96 


1.83 


3.92 


4 33 


5.83 


Post 


.44 


1.89 


1.99 


4.05 


2.19 


5.33 


Delaiyec 


75 


2.38 


1.92 


5.53 


2.77 


8.06 


Pre 


.35 


1.48 


.83 


2.21 


1 75 


4 36 


Post 


2.15 


2.31 


3.16 


3.86 


.50 


1.61 


Delayec 


2 13 


2.42 


2.08 


2.92 


.54 


1.67 


Pre 


6.97 


8.75 


6.54 




5 57 


7 77 


Post 


7.80 


8.01 


12.09 


8.42 


6.34 


8.11 


Delayec 


15.75 


8.72 


8.93 


7.13 


13.03 


14.40 


Pre 


1.43 


3.06 


3.87 


5.07 


2.24 


3.93 


Post 


2.19 


3.89 


2.27 


2.82 


.97 


2.76 


Oelayec 


1.08 


1.84 


2.77 


3.79 


.53 


2.24 


Pre 


1.51 


1.78 


1.05 


1.11 


1.12 


1.33 


Post 


1.55 


1.43 


1.55 


2.31 


1.53 


1.61 


Deleiyec 


1.01 


.66 


1.56 


2.00 


.86 


.91 
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TABLE M (cont'd) 



Direct Instruction 
Behaviors 


Principal Invol^nenl 
Group 

M SD 


Regular Inservlci 
Group 

M SD 


Control Group 
M SD 


Monitored Seatwork 


Pre 


14.06 


11.76 


18.33 


13.22 


14.44 


12.23 




Post 


8.80 


3.90 


8.53 


4.87 


12.72 


11.16 




De1«iyec 


7.81 


3.43 


12.91 


10.20 


12.23 


10.71 


UrwDnltored Seatwork 


Pre 


5.96 


7.89 


5.15 


6.67 


7.94 


10.16 




Post 


6.30 


11.79 


1.63 


1.85 


8.09 


9.94 




Del^ 


.55 


.85 


2.29 


3.16 


4.49 


9.28 


Check Seatwork 


Pre 


2.01 


5.88 


1.08 


2.17 


.90 


• 2.93 


at End 


Post 


1.07 


1.39 


.83 


1.00 


.96 


1.58 




Dele^yec 


1.69 


1.67 


.88 


1.41 


1.53 


3.15 


Length of Lesson 


Pre 


39.23 


6.68 


43.81 


8.30 


43.82 


8.62 


Post 


42.14 


7.16 


43.72 


10.25 


42.71 


11.42 




Del^yec 


39.61 


6.93 


42.25 


7.53 


40.97 


6.91 



lesson, however, the pattern was just the reverse: the principal Involvement 
group spent much more time than the regular Inservlce group on developmental 
Instruction, and much less time In unmonltored seatwork. 

The data were analyzed to determine whether training effects were consistent 
across districts. Data from the two small districts were combined, while the 
large district was treated separately. It was possible to combine data for the 
two small districts because the Instructional behavior of teachers in these 
districts was similar. This judgement was based on Inspection of Table A-1 (see 
Appendix J), which shows the length of tim that teachers In each district used 
each of the 13 Instructional be' -lors. 

Table A-2 In Appendix J presents district-level results for average number 
of Instructional behaviors used for criterion time periods by each treatment 
group. The effects observed In the total sample were generally replicated 
within districts. Both trained groups outperformed the control group at each 
time of observation, and the principal Involvement group outperformed the regu- 
lar Inservlce group In the delayed lesson. 

Table A-3 presents the percentage of teachers In each treatment group using 
each Instructional behavior for the criterion time period In District 1 and In 
District 2-3. Table A-4 presents the same breakdown for the time duration of 
each behavior. The results shown in these tables Indicate that training effects 
were generally consistent across the district. 

Summary . The primary variable used to test hypotheses 1 and 2 was the 
number of Instructional behaviors that teachers used to criterion level. 
Hypothesis 1 was strongly supported; the combined trained groups used more of 
the techniques than the control group In the post-treatment and delayed lesson. 
Hypothesis 2 was partially supported, since teachers in the principal Involve- 
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ment treatment used more of the behaviors than teachers In the regular Inservlce 
treatment In the delayed lessons. The two trained groups performed similarly In 
the post-treatment lesson. 

Hypothesis 3. Students of teachers who participate In the staff development 
program (with or without principal Involvement) will earn higher scores on 
mathematic s achievement tests than students of teachers who do not 
participate In the program. ... - 

Hypothesis 4. Students of teachers who participate In the staff development 
program with principal Involvement will earn higher scores on mathematics 
achievement tests than students of teachers who participate In the program 
without principal Involvement . 

These hypotheses were tested using two different measures of student mathe- 
matics achievement — a curriculum-referenced test and a nationally standardized 
test. 

Curriculum-referenced achievement test. This test was administered to stu- 
dents of all participating teachers before and again several months after the 
staff development program. Descriptive statistics for total scores of each 
treatment group are shown in table 4-5. The pretest achievement level of 
students whose teachers were in the principal Involvement treatment was substan- 
tially lower than students who:>.^ teachers were in the other treatment con- 
ditions* 

On the posttest administration the students in the principal involvement 
treatment were still achieving at a lower level, but the gap between their per- 
formance and the performance of -tudents In the other treatments had narrowed. 
Analysis of mean pretest-posttest gains in table 4-5 indicates t'hat students in 
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TABLE 4-5 

Descriptive Statistics for Performance of Each 
Treatment Group on the Curriculum-Referenced Test (Total Score) 

Principal Involvement Regular Inservlce Control 
Variable Group (N*16) Group (N>16) Group (N»18) 

N (SO) N (SO) M (SD) 

Pre 33.69 (9.16) 39.61 (6.77) 38.75 (4.87) 

Post 41.22 (8.64) 44.93 (9.07) 44.39 (5.40) 

Gain 7.53 5.32 5.64 
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the principal Involvement treatment made the largest gain of the three treatment 
groups. The gains of students In the regular Inservlce group and control group 
were similar. 

The analysis of covarlance summary for trea^..ft t group differences on the 
posttest administration of the curriculum-referenced test Is shown In table 4-6, 
The adjustment: procedure was found to be Invalid since both the homogeneity of 
regression assumption and homogeneity of variances assumption were violated. 
Therefore, an alternative statistical test of hypotheses 3 and 4 was made. The 
jt test for independent means was used to determine whether the gain scores of 
the contrast groups differed significantly from each oth*^r. 

The results of the ^ tests are shown In table 4-7. Although the mean gain 
of the trained groups Is higher than that of the control group, the difference 
Is not statistically significant. The difference between the mean gains of the 
principal Involvement and regular Inservlce group almost reaches statistical 
significance (p « .07). Also, it should be noted that the principal involvement 
group outg?1ned the regular inservlce group by approximately the same margin 
that it outgained the cor* ol group. 

Table A-5 in Appendix J presents descriptive statistics for the 
curriculum- referenced test for each district and grade level combination. The 
principal Involvement group outgained the regular inservlce group at both grade 
levels in District 1, and at the fourth-grade level in Districts 2 and 3. Also, 
the principal Involvement group outgained the control group at both grade levels 
in Districts 2 and 3, and at the fifth-grade level in District 1. 

Standardized achievement tast. The California Achievement Test (CAT) was 
administered in each district before and after the staff development program. 
Standard score equivalents (NCE scores) for the computation subtest, concepts 
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TABLE 4-6 

Analysis of Ccvarlance Summary for Curriculum-Referenced Post Test (Total Score) 

Source df MS F p 

Regression (b » .81) 1 1847.73 84.83 .001 

Treatment 2 9.74 .45 NS 

Within 46 21.78 

Homogeneity of regression 3.54 .04 

Homogeneity of variances (pre) 3.03 .05 

Homogeneity of variances (post) 2.31 .10 
Adjusted means 

Principal Involvement (PI) 44.46 

Regular Inservlce (RI) 42.92 

Control (C) 43.15 

PI and RI vs C t - .39 p « NS 

PI vs RI t « .88 p • NS 
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TABLE 4-7 



Contrast 



Contrasts of Treatment Groups on Curriculum-Referenced 

Test G4ln Scores 



N SP 



Trained Groups 32 6.43 5.02 

vs 

Control Sroup 18 5.64 3,53 



Principal Involvement Group 16 7.53 5.43 

vs 

Regular Inservlce Group 16 5.32 3,49 
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,59 NS 



1.42 .07 



subtest, end total test wore used In the statistical analyses. 

The performance of the three treatment groups on these measures Is shown In 
table 4-8. The groups were approximately at the national average for the total 
CAT test on the pretest administration. All groups made gains, with the regular 
Inservlce group making the largest gain and with both trained groups outgalning 
the control group. The same pattern of relative gains appears In the two sub- 
tests. However, the differences In gains on the concepts subtest are slight. 
The greatest difference In gains Is found on the computation subtest, on which 
the regular Inservlce group outgalned the other two groups. 

Table 4-9 shows the analysis of covarlance summary for the CAT total score* 
Neither of the planned contrasts (trained groups versus control groups; prin- 
cipal Involvement group versus regular Inservlce group) 1s statistically signi- 
ficant. However, the critical assumption of homogeneity of rearesslon was not 
' satisfied (p - .03). 

Similar prcblems occurred In the analysis of covarlance for the two sub- 
tests. A significant effect for homogeneity of regression (p ■ .04) and homoge- 
neity of pretest variances (p * .02) was found on the computation subtest. The 
homogeneity of regression effect approached significance (p « .12) and the homo- 
geneity of pretest and posttest effects were significant (p « .05) on the con- 
cepts subtest. Therefore, hypotheses 3 and 4 were tested using the gain scores 
of each treatment group. 

Table 4-10 presents the results of t^ tests for the contrasts of interest. 
None of the hypotheses were supported. A noteworthy result Is that the regular 
inservlce group outperformed the principal Involvement group by a substantial 
margin on the computation subtest. This difference In achievement gain Is oppo- 
site to that which was predicted. 
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TABLE 4-8 



Descriptive Statistics for Performance of Each 
Treatment Group on the California Achievement Test 



Variable 



Computation 



Concept 



Total 



Principal Involvement 
Group (N-18) 



M 



(SD) 



Pre 48.80 
Post 56.65 
Gain 7.80 



11.12) 
11.02) 



Pre 50.53 ( 8.60) 
Post 55.23 ( 8.27) 
Gain 4.70 



Pre 48.64 
Post 55.42 
Gain 6.78 



.75) 
.28) 



Regular Inservlce 
Group (N«16) 



M 



(SO) 



49.52 (5.94; 
61.09 (7.98] 
11.57 



53.61 (4.30) 
58.50 (4.24) 
4.89 

51.93 (5.16 
60.29 (5.65 
8.36 



Control 
Group (N«18) 



M 

50 
58 



(SO) 

.37 (6.18) 
.07 (9.13) 



7.70 



51.57 (5.60) 
55.37 (6.22) 
3.80 



50.86 
56.63 (7 
5.77 



(6.15) 
.63) 
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TABLE 4-9 

Analysis of Covarlance Summary for 
the California Achievement Test (Total Score) 



Source 



df 



MS 



Regression (b • .64) 

Treatment 

Within 



1 
2 

46 



Homogeneity of regression 2 
Homogeneity of variances (pre) 
Homogeneity of variances (post) 

Adjusted means 

Principal Involvement (PI) 56.73 
Regular Inservlce (RI) 59.23 
Control (C) 56.35 
PI and RI vs C t « .93 p » NS 
PI vs RI t - 1.16 p NS 



1119.44 
39.67 
35.59 
120.95 



31.46 
1.11 

3.81 
2.20 
1.71 



.001 

NS 

.03 

NS 
NS 
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TABLE 4-10 

Contrasts of Treatment Groups on California 
Achievement Test Gain Scores 

TOTAL TEST GAIN SCORE 



Contrast 


N 


M 


SD 


t 


P 


Trained Groups 


32 


7.58 


5.74 






vs 








1.03 


NS 


Control Group 


18 


5.71 


7.00 






Principal Involvement Group 


16 


6.78 


3.55 






vs 








.81 


NS 


Reoular Inservlce Grouo 


16 


8.38 


7.35 








COMPUTATION SUBTEST GAIN SCORE 






Cor»trast 


N 


M 


SD 


t 


P 


Trained Groups 


34 


9.61 


8.4^ 






vs 








.80 


NS 


Control Group 


18 


7.70 


8.09 






Principal Involvement Group 


18 


7.86 


6.13 






vs 








.1.35 


.90 


Regular Inservlce Group 


16 


11.58 


10.30 
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No t, tests were done for contrasts Involving the concepts sul'^^il»as the 
criterion, since the difference between treatment groups In mean gain was slight 
(see table 4-8). f 

Tables A-6 and A-7 In Appendix J present descriptive statistics tSV CAT 
total score and computation score, respectively, for each dlsti^jyifid arade 
level combination. The pattern of withln-dlstrlct results Is Silferlilly con- 

V 

't ' 

si stent with the results observed In the total sample. 

Summary . Hypotheses 3 and 4 were tested using both a currlcuJuBFSreferenced 
test and a nationally standardized test (CAT) of mathematics achievement. With 
respect to hypothesis 3, the trained groups made greater galn^j^il.thy control 
group on both tests, but the differences were slight and no^i^tlstlcally 
significant. With respect to hypothesis 4, the principal Involvement group maae 
a greater gain than the regular Inservlce group (p » .07) on the curriculum- 
referenced test, as predicted. The regular Inservlce group, however, made a 
greater gain than the principal Involvement group on the comgutatlon subt^t of 
the CAT. This result Is opposite to the hypothesized difference. The two 
trained grqjiips did not differ significantly from each other on the CAT total 
test or the CAT concepts test. 

Hypothesis 5. Students of teachers who participate In the staff development 
program (with or without principal Involvement) will exhibit Jjjs s off-task 
behavior during Instruction than students of teachers who d o no t 
participate In the program. 

Hypothesis 6. Students of teachers who participate In the staff development 
program with principal Involvement will exhibit less off-task behavior 
than students of teachers who participate Jn. the program without 
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principal Involvement* 

The number of off-task students was counted at five-minute Intervals for 
each observed mathematics lesson. The average number of off>*task students 
across the intervals was then computed. The results for each treatipent group 
on this measure Is shown In table 4-11. The average class size for each treat- 
ment group Is also shown In this table. Since class size varied across groups. 
It was necessary to compute for each class the average number of off-task stu- 
dents as a percentage of the total number of students In the class. Table 4-U 
presents descriptive statistics for this variable, which Is labelled "percentage 
of off-task students*" 

The three treatment groups were similar to each other In percentage of off- 
task students In the pretreatment lesson* Both trained groups show^ a slight 
reduction In off-task percentage on the post-treatment lesson, while the control 
group remained unchanged. On the delayed lesson, more than one- fourth of the 
average control class was off-task* The percentage of off-task students In the 
two trained groups was well below this level* 

Table 4-12 reports the tests of hypotheses 5 and 6 at two times of obser- 
vation. The assumpffons underlying the adjustment procedure were satisfied with 
one exception. The treatment group variances differed significantly (p « .006) 
In the delayed lesson. Violation of this assumption, however. Is minimal when 
the sample size for each group and treatment group variances on the pretest are 
similar (Elashoff 1969, p. 395). 

Hypothesis 5 was supported since trained teachers had a lower percentage of 
off-task students than did control teachers in both the post lesson (p =r>10). 
and the delayed lesson (p * .05). Hypothesis 6 was not supported slntfs^t " 
principal Involvement group and the regular 1nsef^|^ teachers did not d||?eK 
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TABLE 4-11 

Descriptive Statistics for Student Off-Task Behavior 



Principal 

Involvement Regular Inservlce Control 

Variable Group (N«18) Group (N«17) Group (N-18) 

M (SD) M (SD) M (SD) 

Number of Off-Task Pre 3.98 (2.85) 4.?3 (2.51) 4.11 (2.56] 

Students Post 2.67 (2.33) 3.44) (2.80) 3.76 (1.79 

Delayed 3.67 (2.49) 3,39 (1.60) 5.02 (3.75] 



Number of Students Pre 20.31 (6.38) 22.06 (3.19) 20.94 (4.63] 
in Class Post 19.94 (6.42) 21.29 |3.80) 20.14 (4.29; 



Delayed 19.56 (5.63) 21.47 (3.83) 19.53 (7.03 



Percentage of Off-Task Pre 18.54 (13.46) 18.74 (9.60) 19.40 (11.45] 
Students Post 13.73 11.92) 16.18 (13.42) 19.40 (9.82 

Delayed 18.87 (11.99) 16.05 (7.89) 24.33 (18.19; 
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TABLE 4-12 

Analysis of Covarlance Summary for Percentage 
of Off-Task Students 

POST TREATMENT MEANS 

df MS F 



Regression (b - .22) 

Treatment 

Within ^ 



1 
2 
49 



.03 
.01 
.01 



Homogeneity of regression 
Homogeneity of variances (pre) 
Homogeneity of variances (post) 
Adjusted means 

Principal Involvement (PI) 
Regular Inservlce (RI) 
Control (C) 

PI and RI vs C t » 1.27 
PI vs RI t » .61 



.14 
.16 
.19 



p - .10 

p » NS 

DELAYED TREATMENT MEANS 



Regression (b * .19) 

1 . eatment 

Within 



1 

Z 

48 
2 



Homogeneity of regression 
Homogeneity of variances (pre) 
Homogeneity of variances (post) 
Ad justed means 

Principal Involvement (PI) 
Regular Inservlce (RI) 
Control (C) 

PI and RI vs C t « 1.71 
PI vs RI t « .64 p « NS 



.03 
.03 
.02 
.01 



18.94 
16.08 
24.23 
3 « .05 
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2.41 
1.00 

.35 
.90 
.78 



1.71 
1.65 

.66 
.80 
5.16 



NS 
NS 

NS 
NS 
NS 



NS 
NS 

NS 
NS 
.006 
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significantly In percentage of off-task students In either lesson. 

Table A-8 in Appendix J presents the S4me descriptive statistics for off- 
task behavior as In table 4-11, but separately for District 1 and Districts 2-3. 
It appears that there is a district- avel effect for percentage of off-task stu- 
dents. In District 1 the two trained groups of teachers lowered their percen- 
tage of off-task students in the post lesson and maintained this gain in the 
delayed lesson. In Districts 2 and 3, the trained groups made slight gains in 
the post lesson but lost them in the delayed lesson. Thus, the support for 
hypothesis 5 in the total sample is largely a function of a training effect that 
occurred in District U Hypothesis 6 was not supported in either District 1 or 
Districts 2-3. 

Summary. The variable used to test hypotheses 5 and 6 was the percentage of 
off-task students in the post and delayed lessons. Hypothesis 5, involving the 
contrast of the trained groups and the control group, was supported. Hypothesis 
6, involving the contrast of the two trained groups, was not supported. 

Supplementary Analyses 

Treatment Implementation 

The report of Good and Grouws' fourth-grade experiment provides data on 
teachers' implementation of their instructional model (Good and Grouws 1979, 
p. 358), The present study collected data on some of the same implementation 
variables. 

Table 4-13 presents comparisons between the two studies on similarly 
measured variables. Good and Grouws' report includes only post-treatment data, so 
table 4-13 does not contain pre-treatment data from the present study. 

Comparison of the control groups in the two studies indicates that their 
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TABLE 4-13 

Comparison of Training Effects In Good and Grouws' 
Fourth-Grade Experiment and In the Present Study 



Variable 

1. Conduct review 

2. Check homework 

3. Mental computation 

4. At least 5 minutes 
on development 

5. Controlled practice 



6. Hold accountable 
for seatwork 

7. Seatwork directions 
longer than 1 minute 

8. Assign homework 



Percentage of Teachers 
G and 6 Principal Regular G and G Present 
Treatment Involvement In service Control Control 



Post 


91% 


61% 


53% 


62% 


67% 






56% 


65% 




22% 


Post 


79% 


72% 


88% 


20% 


0% 


Delayed 




83% 


71% 




11% 


Post 


69% 


61% 


76% 


6% 


11% 


Delayed 




56% 


53% 




11% 


Post 


45% 


61% 


82% 


51% 


39% 


Delayed 




89% 


76% 




50% 


Post 


33% 


39% 


59% 


20% 


17% 


Delayed 




33% 


41% 




6% 


Post 


59% 


72% 


59% 


31% 


44% 


Delayed 




78% 


53% 




50% 


Post 


18% 


50% 


47% 


23% 


56% 


Delayed 




33% 


53% 




28% 


"nst 


66% 


78% 


65% 


13% 


6% 


uiilayed 




78% 


71% 




6% 
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level of use of Good and Grouws' Instructional model was similar. The only 
noteworthy differences were In conducting reviews (delayed lesson) and In giving 
seatwork directions. On the latter variable, the actual mean length of time for 
giving directions in the control group post lesson In the present study was 1.53 
minutes. Thus, this group was not much different In performance from the 
control group 1n Good and Grouws' study. 

The performance of trained teachers was also similar across the two studies. 
The only noteworthy differences were on the variables of review, development, 
and seatwork directions. Trained teachers In Good and Grouws* study achieved 
better Implementation of review and seatwork direction procedures, while 
teachers In the present study achieved better Implementation of the time cri- 
terion for developmental instruction. 
Correlation Between Variables 

Table 4-14 shows the correlations between implementation and off-task 
variables. The correlations between the three measurements of implementation 
(i.e., the number of techniques used for the criterion time level) are just 
slightly positive. It appears that individual differences in implementation are 
relatively unstable from one time of observation to the next. The correlations 
between the three measurements of student off-task behavior are even lower, 
indicating that this classroom behavior was quite unstable across the thr>?e 
observed lessons. 

An interesting result in table 4-14 is the consistent negative correlation 
between teachers* implementation of Good and Grouws* instructional model and 
student off-task behavior. It appears that implementation of the model was 
associated with a decrease in student off-task behavior, especially in the post 
lesson (r =» -.53). This effect of Good and Grouws* staff development program 
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Imp] ement 
Pre 

Implement 
Post 

Implement 
Delay 

Off Task 
Pre 

Off Task 
Post 

Off Task 
Delay 



TABLE 4-14 

Correlation Matrix of Implementation 
and Off-Task Variables^ 



Implement Implement Implement 
Pre 



Post 
.27 



Delay 
.20 

•41 



Off Task 
Pre 

-.27 
-.15 
-.04 



Off Task 
Post 

-.36 
-.53 
-.35 
.22 



Off Task 
Delay 

-.21 
-.12 
-.34 

.19 

.20 



Note. — N for all correlations Is 53, except for Off-Task, Delay (N»52) 

1 The Implementation variable Is the number of techniques that a teacher used 
for the criterion time level. The off-task variable Is the average percentage 
of off-task students In a lesson. 
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has not been previously Investigated. 

Table 4-15 reports the IntercorreTatlons of the achievement test variables. 
As expected, there was a high correlation between the pre and post administra- 
tions of the curriculum-referenced test (£ * .75) and between the pre and post 
administrations of the California Achievement Test (£ • .65). The lower pre- 
post correlation for the latter test may be the result of the longer time lag 
between the two administrations and the fact that a different version of the 
test was administered on each testing occasion. 

Although the two tests are correlated, the degree of association Is not high 
(£ « .54 for total post score). It appears that, to a large extent, the 
curriculum-referenced test and the nationally standardized test measure dif- 
ferent aspects of mathematics achievement. 

Analyses of the relationship between teachers' Implementation of Good and 
Grouws' instructional model and student achievement gains are In progress. 
These analyses will be presented In a subsequent report. 
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%. , TABLE 4-15 ^ 

Correlation Matrix of Curriculum-Referenced Test 
Variables and California Achievement Test Variables 

Curric Curric Comput Comput Concepts Concepts Total Total 
Pre Post Pre Post Pre Post Pre Post 

Referenced Test — .75 .27 .38 .40 .40 .41 .45 

Pre 

Referenced Test — .27 .49 .39 .50 .38 .54 

Post 

Computation Subtest .57 .80 .60 .94 .58 

Pre 

Computation Subtest .49 .79 .53 .95 

Post 

Concepts Subtest — •72 .95 .64 

Pre 

Concepts Subtest — •^^ 

Post 

CAT Total — 
Pre 

CAT Total 
Post 

N's for correlation coefficients vary between 49 and 51. 
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CHAPTER FIVE 

Discussion vf 
This chapter Is organized Into four sections. In the first section we 
discuss the meaning of observed differences In the effects of the principal 
involvement and regular inservlce treatments. Do the results demonstrate that 
the Involvement of a building principal can enhance the Impact of a staff deve- 
lopment program on his or her teachers? The second section of the chapter con- 
siders the significance of the findings concerning the relative effectiveness of 
the staff development treatments (Ignoring the factor of principal Involvement) 
and the control treatment. 

In the third section we draw upon the study's results to reconsider the 
leadership functions that a principal, or other designated person, can perform 
in promoting the implementation of a staff development program of the type used 
in this study. Next, we discuss the implications of the study's results for 
school organization and school Improvement. Thre chapter concludes with a set of 
recommendations for research on instructional leadership. 

Principal Involvemen t in Teachers' Staff Development 
The major purpose of this study was to determine the effects of directly 
involving elementary school principals in their teachers' staff development 
activities. Previous research suggests that this type of Involvement charac- 
terizes principals of effective schools. The research, however, consists 
entirely of descriptive and correlational studies, which are weak designs for 
determining whether observed relationships between principal behavior and 
teacher and student performance are causal. An experimental design, in which 
principal behavior is manipulated by intervention, is necessary for determining 
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whether principal behavior directly affects teacher Instruction and student 
achievement. In the present study the manipulation was effected by having one 
group of principals participate In the teachers* staff development program (the 
principal Involvement treatment) while withholding participation from another 
group of principals (the regular inservlce treatment). 

Effects on Teacher Instruction* Teachers In both treatment groups Imple- 
mented Good and Srouws' Instructional model to a nearly Identical level In the 
post lesson. In the delayed lesson, however, teachers In the principal Involve- 
ment treatment Increased their Implementation of the Instructional model, while 
teachers in the regular Inservlce treatment regressed slightly. The difference 
between treatment groups on the Implementation measure approached statistical 
significance (p « .07), ' - 

Why were differences In Implementation not observed until the delayed 
lesson? Two plausible explanations are suggested by the way In which the 
experimental events were timed. The first explanation Involves the timing of 
the principal Involvement treatment. The principals In this treatment attended 
the second staff development session In mid-February and their first Instruc- 
tional leadersnip session at the end of February. They were given until April 1 
to complete their first observatlon-and-conference. The post lesson obser- 
vations, however, occurred between March 8 and March 18. Thus, the principals 
may have had too little time to exert Influence on the teachers' post lesson 
instruction. 

The situation at the time of the delayed lesson was different. All of the 
principals in the principal involvement treatment had completed their first 
observation-and-conference by April 1, and several had completed a second. 
Also, all of the principals attended a second Instructional leadership meeting 
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at the end of April. Since the delayed lesson observations occurred between May 
3 and May 13, there was ample time for principals to exert their Influence on 
teachers' Implementation of the Instructional model. Their Influence had the 
desired Impact, as reflected In the difference In Implementation scores of 
teachers In the two trained groups. 

The second explanation for the absence of a principal Involvement effect 
until the delayed lesson Is that the experiment commenced several months after 
the school year began. Both principals and teachers commented that homework and 
whole group Instruction are difficult practices to Implement once the pattern of 
instruction Is set In September. For example. In Chapter 3 we referred to the 
principal who stated that teachers who had practiced some form of Individualized 
Instruction In the first semester were hard pressed to alter their practices In 
February simply on the basis of two brief training sessions In Good and Grouws' 
model. (In this respect It Is Interesting to note that the training In Good and 
Grouws' fourth-grade study occurred during the first month of the school year.) 

Although principals generally had a positive reaction to the Instructional 
model, they and their teachers needed time to think through procedures for 
shifting Instructional practices. The homework policy In the Instructional 
model challenged the participants, but most of them confronted It In the first 
staff development session, and they began implementation by the second session. 
In both trained groups the implementation level for homework assignment 
increased dramatically from the pre lesson to the post lesson, and stayed at the 
same level in the delayed lesson (see table 4-3 ). 

The recommendation for whole group instruction in the model was much more 
difficult to implement. Individualized instruction was entrenched in many of 
the classrooms. Principals and teachers found it difficult to think of ways to 
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handle Individual differences among students — especially low ability students 
" In the context of whole group Instruction. The teachers manual provided 
little Information about ways to handle this problem. When the research staff 
visited principals and teachers In the principal Involvement treatment In March 
and April, these groups reported some success In Implementing whole group 
Instruction. This gradual success Is reflected In the observational data. 
Teachers In the principal Involvement treatment spent about eight minutes on 
development In the post lesson and almost twice that amount of time In the 
delayed lesson (see table 4-4). According to the scoring system, teachers are 
not given credit for development unless the teacher Is engaged In this Instruc- 
tional mode with the whole or nearly the whole class. Also, these teachers 
greatly reduced the amount of time they spent i jeatwork (see table 4-4), which 
Is another'indlcator of a shift away from Individualized Instruction. 

The finding of a treatment group difference In the delayed lesson, but not 
In the post lesson, was unanticipated. Also, Implementation was only measured 
at two points In time following training. Despite these problems, the post hoc 
Interpretation offered above makes good sense. Principals need time to work 
with teachers \.o Influence their Instruction. Principals In the Involvement 
treatment had adequate time by the delayed lesson, but not by the post lesson. 

Effects on student achievement (curriculum-referenced test). Students in 
the principal Involvement treatment made greater gains on the curriculum- 
referenced test of mathematics achievement than did students In the regular 
Inservlce treatment. The difference was statistically significant at the .07 
level (one-tailed test). Several explanations for this result seem plausible. 

The superiority of students In the principal Involvement group on the 
curriculum-referenced test Is explained most directly by their teachers' imple- 
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mentation of Good and Gpouws' Instructional model. In the delayed observation, 
teachers In the principal Involvement group spent approximately 16 minutes of 
the lesson on developmental Instruction, whereas teachers In the regular Inser- 
vlce group spent only 9 minutes on this activity (see Table 4-4). In other 
words, teachers who had Involved principals spent more time engaged In Instruc- 
tion of new material. Since the curriculum-referenced test was designed specifi- 
cally to measure amount of curricuum covered and mastered, it seems reasonable 
that students whose teachers spent more time on developmental instruction would 
perform better on the test than students whose teachers spend less time on deve- 
lopmental instruction. 

This explanation of treatment group differences on the curriculum-referenced 
test suggests that the Involved principals affected student achievement by pro- 
moting teacher implementation of critical aspects of Good and Grouws' instruc- 
tional model. Teachers whose principals were not Involved in the staff 
development program did not have a person in authority encouraging them to 
Implement ~ and to continue implementing — these critical aspects. 

Another explanation of the curriculum-referenced test results is that the 
Involved principals exerted Influence by encouraging teachers to improve student 
performance on the post-administration of this test. The involved principals in 
the two small districts had worked with this test for several years and so were 
quite familiar winh its format and meaning. The involved principals in the 
large district had not previously experienced this test. As mentioned in 
Chapter 3, these principals were given information about the test in their ini- 
tial instructional leadership session in February. Peer pressure might have 
operated to Influence the Involved principals to work with teachers so that 
their school would produce favorable gains on this test. 
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The Involved principals were further sensitized to the curriculum-referenced 
test when the results of the pretest were given In late February to each par- 
ticipating teacher for his or her class and to all principals for each grade 
level In their schools. Also, the Involved principals knew that the curriculum- 
referenced test would be re-administered In May to determine student achievement 
gains. All of these sensitization procedures could have Influenced the Involved 
principals to help teachers enhance student performance on math concepts and 
procedures measured by the curriculum-referenced test. 

It can be argued that principals In the other two treatment groups were 
similarly sensitized because ;hey too received grade-by-grade results for the 
pretest, and each teacher under their supervision received pretest results for 
his or her classroom. No doubt these procedures resulted In some sensitization, 
but two Important differences between these groups of principals and the 
Involved principals should be noted. First, only the Involved principals 
attended the training sessions In Good and Grouws' program, at which tirje '.hey 
learned that student mathematics achievement could be affected by having 
teachers make changes In their Instructional behavior. Although this point 
seems obvious, participants In the sessions had not known previously about the 
type of correlational process-product research on which Good and Grouws' program 
and similar programs are based. 

The second difference between the Involved principals and other principals 
is that only the former group had the opportunity to meet together (In the first 
leadership session) to discuss the VEC test results and their meaning In rela- 
tion to Good and Grouws' Instructional model. Thus, type and frequency of sen- 
sitization were substantially different for Involved principals than for 
principals in the regular Inservlce and control treatments. 
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In summary, the above discussion suggests tfiat the involved principals could 
have Influenced the results on the curriculum-referenced test In two ways. One 
type of Influence Involves helping teachers to Implement critical aspects of 
Good and Grouws* Instructional model. The other type of Influence Involves 
encouraging teachers to help students Improve their achievement test perfor- 
mance. These two types are not mutually exclusive. There Is evidence that the 
Involved principals could have used one or both types of Influence. 

The treatment group differences on the curriculum-referenced test can be 
attributed to other factors than principal Influence. It must be noted that tie 
pretest achievement of students in the principal Involvement treatment was 
substantially lower than that of students in the regular inservice treatment 
(see table 4-5). The Involved principals undoubtedly were aware that their 
schools had lower-achieving students than other schools in the district. This 
disparity may have stimulated the Involved principals to devote extra effort 
working with teachers to reduce the disparity. 

Still another explanation for the treatment effect Involving the curriculum- 
referenced test is that it is an artifact of the statistical procedure used to 
determine the presence of the effect. As shown in table 4-5, students in the 
principal Involvement treatment made a 7.5 point mean gain on this test, 
starting from a pretest mean score of 33.7. The gain of the other two treatment 
groups was not quite so large: they gained approximately 5.5 points, starting 
from a mean of approximately 39 points. With students starting from a lower 
initial achievement level, teachers and principals in the principal Involvement 
treatment may have found it easier to stimulate Improved achievement than was 
the case for teachers and principals working with students who started at a 
higher initial achievement level. This argument, however, can be reversed. It 
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may actually be more difficult to help lower-achieving students make appreciable 
learning gains because they lack motivation and a belief that they can ao so. 
Without evidence to the contrary, this argument seems as plausible as Its con- 
verse. 

The different levels of pretest achievement would further complicate 
Interpretation of treatment effects If the posttest measure of achievement had a 
low celling. The results shown In table 4-5, however. Indicate that the two 
treatment groups with higher initial achievement scored approximately 45 points 
on the posttest. Since the posttests for grades 4 and 5 contained 65 and 70 
Items, respectively. It seems unlikely that achievement gains for these two 
groups were depressed by a celling effect. 

An Interaction between pretest achievement level and achievement gain cannot 
be ruled out as an explanation of the observed difference In gain between stu- 
dents In the.,pfti^nc1pal Involvement treatment and students In the regular Inser- 
vlce treatment. Thus, this explanation stands as a plausible alternative to the 
explanation that the observed differences occurred because principal Involvement 
augments the effects of a staff development program. 

Effects on student achievement (standardized test). Students whose teachers 
were In the regular inservlce treatment made greater gains on the computation 
scale of the nationally standardized test than did students whose teachers were 
In the principal Involvement treatment. How can this result be explained, espe- 
cially In view of the fact that the same group of students had lower performance 
on the curriculum-referenced test? 

The explanation we favor Is that participating teachers emphasized aspects 
of the mathematics curriculum to which they were sensitized In their particular 
treatment condition. Support for this explanation comes from two sources. 
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First, the results shown In table 4-15 Indicate that the curriculum-referenced 
test and the computation test measured substantially different aspects of the 
mathematics curriculum. The correlation between the pretests was only ,27, and 
the correlation between the posttests was .49. The percentage of shared 
variance was much less than the percentage of Independent variance. If teachers 
In the two treatment's were emphasizing different aspects of the mathematics 
curriculum, this would likely result In differential performance of their stud- 
ents on the two tests. 

The instructional model presented 1n the staff development program was not 
curriculum-specific. An examination of the teachers manual reveals that It 
refers to Instruction In math "concepts," "skills," and "problem-solving." 
Almost without exception, however, the examples refer to computational skills. 
In the main part of the manual, 18 of the 19 examples involved the basic com- 
putational skills of addition, subtraction, multiplication, and division. In the 
appendix, 42 of the 46 examples Involved the same computational skills. 

This analysis of the manual suggests that It would be easy for teachers to 
get the Impression that the purpose of the Instructional model was to Increase 
students' proficiency 1n basic computational skills. Also, the substantive con- 
tent of the two staff development sessions was focussed almost entirely on the 
Instructional model rather than on curriculum Issues. In ths absence of other 
Input, teachers In the regular Inservlce treatment might well have concentrated 
on improving students' computational skills rather than on Improving their per- 
formance across the varied topics covered In the fourth- and fifth-grade mathe- 
matics curriculum. 

Evidence In support of this argument comes from differential treatment group 
Implementation of the Instructional model. We have already discussed the fact 

111 

105 



that the regular Inservlce teachers spent less time on developmental Instruction 
In the delayed observation lesson. Also, these teachers spent substantially more 
time on seatwork than did teachers in the principal Involvement treatment (see 
table 4-4), The former group spent a total of 17 minutes on seatwork directions 
and seatwork, whereas the latter group spent a total of 9 minutes on the same 
activities. 

Seatwork Is typically used for practice of computational skills. (The 
observers noted virtually no Instruction or practice Involving mathematical 
problem-solvlno.) Since the regular Inservlce group spent more time on this 
activity, It Is reasonable to expect that their students would score higher on a 
measure of computational skill. 

This explanation suggests that the regular Inservlce teachers were trying to 
Implement ♦he Instructional model, but primarily those aspects that are oriented 
to curriculum objectives Involving computational skills. The teachers with 
Involved principals were also trying to Implement the model, but with more 
emphasis on the total range of curriculum objectives In mathematics. 

Effects on students' off-task behavior. The classes of teachers In the two 
treatment groups did not differ significantly from each other In percentage of 
off-task students. A non-significant difference was found In both the post and 
delayed lessons (see table 4-12). 

Even though no treatment effect occurred for this variable, the finding Is 
Interesting because it supports the Interpretation that we have advanced for the 
effects mentioned above. Reduction of student off-task behavior Is not a major 
goal of Good and Grouws' Instructional model. It Is only discussed In the sec- 
tion on seatwork In the teachers manual (6 pages of the 47 page manual). There 
was some discussion of off-task behavior In the staff development sessions, but 
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It was at best a minor focus. Teachers expressed more concern about students' 
motivation for learning mathematics and for the various activities of the 
Instructional model than they did about whether students were on task from 
moment to moment. Also, there was no explicit attention to off-task reduction 
as a goal In the Instructional leadership sessions for principals In the prin- 
cipal Involvement sessions. 

The fact that off-task reduction was not a highlighted goal of the Instruc- 
tional model could explain why principals and teachers In the two treatments did 
not differ In reduction of off-task behavior. The goals that were differen- 
tially highlighted for the Involved principals — Instructional model Implemen- 
tation and performance on the curriculum-referenced tests — were those for 
which treatment effects were observed. 

Another explanation of the results for off-task behavior Is that the 
observed percentages are at or near the lower limits for this behavior without 
special Intervention being focussed on It. Mclntyre, Copenhaver, Byrd, and 
Norrls (1983) recently determined off-task rates for samples of third-, fifth-, 
and seventh-grade mathematics lessons. The off-task rate was 23 percent, 25 
percent, and 27 percent for the three grade levels, respectively. Fisher and 
his colleagues (1980) observed similar percentages of off-task elementary stu- 
dents In the Begimiing Teacher Evaluation Study. 

In the present study, however, the treatment groups were below this level In 
the prelesson (less than 20 percent) and one treatment group achieved a low off- 
task rate of 13e73 percent after training (see table 14-11). Only one measure- 
ment point (the control group's delayed lesson) approximated the off-task rate 
found In other studies. These comparisons suggest that Instructional management 
In the present sample of classes was better than average. If this was the case. 
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teachers In the trained groups may not have felt a particular need to focus on 
Improving students' on- task rate* 

Magnitude of observed effects* The statlstlcaly significant differences 
between the principal Involvement group and regular Inservlce group were of 
small magnitude. Several features of the experimental situation may explain 
this result. First, the pretest performance of the participating students on 
the nationally standardized test (see table 4-8) Indicates that they were 
approximately at the fiftieth percentile. In contrast, the mean pretest perfor- 
mance of students In Good and Grouws' fourth-grade study was approximately at 

* 

the twenty-eighth percentile on a nationally normed test. It may be that prin- 
cipals In the principal Involvement treatment did not feel the same Intensity of 
need for Improvement of student achievement that participants In Good and 
Grouws* study felt. If this was the case, they would not have attempted to work 
with teachers to bring about the substantial gains In student mathematics 
achievement found 1n Good and Grouws' study. 

Another explanatory factor is the time and duration of the experiment. The 
last measurements of Instructional model Implementation and student achievement 
occurred approximately three months after the staff development session and 
first Instructional leadership session. Differences between the two treatments 
might have been accentuated If subsequent measurements were made after the 
Involved principals had more time to work with their teachers. Also, as we 
Indicated above, the fact that the training sessions occurred In February, well 
after the start of the school year, limited what the involved principals could 
accomplish. 

The intensity of the instructional leadership training may have been another 
factor limiting treatment group differences. The involved principals met 
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together for just twc sessions. Participation In additional sessions might have 
served to Increase their leadership skills and commitment to teachers' Implemen- 
tation of the model. Also, the Involved principals observed most of their par- 
ticipating teachers a total of two times. Additional observations might have 
helped these teachers fine-tune their use of the Instructional model, resulting 
In a higher Implementation level relative to the teachers In the regular Inser- 
vlce treatment. 

Finally, another factor that may have limited treatment group differences Is 
the design of the staff development sessions. It was not feasible to conduct 
separate sessions for teachers In the two treatment groups. Therefore, the 
sessions Included teachers In the regular Inservlce treatment, teachers In the 
principal Involvement treatment, and principals In the principal Involvement 
treatment. The presence of principals from other schools at the sessions may 
have led the regular Inservlce teachers to perceive more need to Implement the 
instructional model than they would have perceived had they attended the 
sessions only with other teachers. 

We do not know the extent to which Initial student ability level and the 
timing and Intensity of training affects the Impact of principal Involvement on 
teachers* staff development. We also do not know whether certain features of 
the experimental design, such as Including both treatment groups In the same 
training sessions, obscured treatment effects. Further research Is needed to 
determine whether principals can be Involved In teachers' staff development so 
that the effects achieve practical significance In addition to the statistical 
significance found in this study. 

Nonrandom assignment of principals to treatments . The sampling unit used in 
the experimental design was the individual school. Thus, the principal and all 
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of the participating teachers In each school were assigned to the same treatment 
group. Because of restrictions Imposed by the school districts, It was not 
possible to randomly assign the schools to treatment groups. 

Because of nonrandom assignment, the possibility exists that some or all of 
the observed group differences on the dependent variables reflect pre-existing 
group differences rather than' treatment effects. As we Indicated In Chapter 2, 
principals were assigned to the principal Involvement treatment because of their 
Interest In the study, their reputation as effective leaders, and/or their 
current workload. If Interest 1n Instruction and effective leadership were 
indeed more prevalent among the Involved principals, these characteristics could 
account for the observed effects favoring the teachers and students of these 
principals. 

The first step In testing the validity of this explanation Is to realize 
that these were characteristics ascribed to the Involved principals by central 
office administrators. Some of the data collected In the experiment can be used 
to check the validity of these ascribed characteristics. For example, table 4-1 
shows the number of Instructional techniques used by teachers In each treatment 
group before the experiment began. The results Indicate very little difference 
between treatment groups. (In fact, the teachers supervised by the Involved 
principals had the lowest Initial level of criterion use of the techniques.) 
Similarly, table 4-11 Indicates very little difference between treatment groups 
in percentage of off-task students at the outset of the experiment. These data 
suggest that the Involved principals were not Initially more effective Instruc- 
tional leaders than principals In the other two treatment groups. 

The issue of treatment group similarity is complicated somewhat by the pre- 
test data on student mathematics achievement. The pretest data for the 
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curriculum-referenced test, shown In table 4-5, clearly Indicate that the 
Involved principals worked In buildings with lower-achieving students. (The 
data for the nationally standardized test, shown In table 4-8, also show the 
same pattern of difference, but the magnitude of the differences Is slight.) It 
can be argued that these principals were Instructional ly effective because their 
teachers' classroom performance was similar to that of principals of students 
who were achieving Initially at a higher level on the currUulum-referenced 
test. 

This argument would be given weight If It could be shown that teachers' 
classroom performance Is affected by student achievement level. The data In 
this study provide only weak evidence that such a relationship exists. The 
correlation between student achievement on the curriculum-referenced pretest and 
pretreatment teacher Implementation of the Instructional model Is only .16. The 
correlation between the former variable and pretreatment percentage of student 
off-task behavior Is only -.22, It seems that teacher and student behavior In 
the classroom Is largely Independent of Initial student achievement level. 
Therefore, If the Involved principals were more effective Instructional leaders, 
they would not have been hampered In affecting their teachers' classroom perfor- 
mance by a lower-achieving student population. 

This analysis suggests, then, that the principals In the three treatment 
groups did not differ Initially In Instructional leadership effectiveness, even 
though they were assigned nonrandomly to treatment groups. Furthermore, to the 
extent that Initial differences existed, they were controlled for their effect 
on the dependent variables by the use of analysis of covarlance or gain score 
analysis. Nonetheless, pre-existing differences between principals In the two 
trained groups cannot be ruled out definitely as an alternative explanation of 



observed group differences on the dependent variables. Further research using a 
better controlled design will be necessary to determine more definitely the true 
Impact of Instructional leadership manipulation on teacher and student perfor- 
mance. 

Concluding comment . Even If other factors operated to affect the observed 
treatment group differences, the weight of the evidence suggests that manipula- 
tion of principal behavior did produce positive effects on teachers and students 
In this experiment. Although It Is Just one experiment. Its results are con- 
sistent with the many descriptive and correlational studies that preceded It. 
Many of these studies (see table 2-6) Identified principals' Involvement In 
teachers' staff development and classroom observation as characteristic of 
effective principals. 

Another characteristic of effective principals Identified In these studies 
is the effort they direct toward the goal of high academic achievement for stu- 
dents. Similarly, the Involved principals In this study were given clear acade- 
mic goals and asked to achieve them. They were asked to help teachers Implement 
an academically-oriented Instructional model and to Improve student achievement 
as measured by a curriculum-referenced test. These are goals that they 
accomplished to a degree. Conversely, the Involved principals did not perform 
differently than the non-Involved principals on goals that were not emphasized 
In the principal Involvement treatment. These goals Involved reduction In off- 
task behavior and computational skill. 

Present and previous research suggest then, that the critical determinants 
of principal effects on teacher Instructional performance and student academic 
achievement are (1) the clear articulation of these outcomes by the principal as 
goals to be achieved and (2) personal effort directed toward these goals. 
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Effectiveness of the Staff Development Program 
The major objective of this study was to determine the effects produced by 
Involving school principals In a staff development program for teachers. A 
secondary objective was to determine the effectiveness of the staff development 
program Itself, Irrespective of the principal Involvement factor. The Inherent 
effectiveness of the program was of Interest to us because It might affect prin- 
cipal Involvement, which was the variable of primary Interest. If the program 
was difficult to Implement and did not produce benefits for students, principals 
In the principal Involvement treatment might choose to become disengaged from 
it Irrespective of their agreement to work on Improving their use of Instruc- 
tional leadership skills. The effectiveness of the program was determined by 
comparing the combined performance of both trained groups (I.e., principal 
Involvement and regular Inservlce) with that of the control group. This com- 
parison constitutes a replication of the design of Good and Grouws' fourth-grade 
experiment. 

The results of these comparisons generally favored the trained groups. 
Their Implementation of the Instructional model was significantly higher (p « 
•001) In both the post lesson and delayed lesson. The results are very similar 
to the results obtained In Good and Grouws' fourth-grade study, as shown by the 
implementation levels for specific behaviors In table 4-13. 

The comparisons Involving student achievement measures were not statisti- 
cally significant. As we discussed above, however, each trained group may have 
emphasized a different aspect of student achievement. If true, comparisons of 
the combined trained groups may be misleading. For each achievement test, the 
control group should be compared with the trained group that emphasized the 
aspect of achievement measured by that test. When this Is done, the results 
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favor the trained groups. The principal Involvement group outgalned the control 
group on the curriculum-referenced test by approximately the same level (1*89 
points) that it outgalned the regular inservlce group (2*21 points)^ Similarly, 
the regular Inservlce group outgalned the control group on the CAT computation 
subtest by approximately the same level (3.87 points) that It outgalned the 
principal Involvement group (3.77 points). While these student achievement 
effects are not as Impressive as those found In Good and Grouws' study, they are 
in the same direction. 

The combined treatment groups achieved a lower percentage of off-task stu- 
dents In both the post lesson (p « .10) and delayed lesson (p « .OS). Although 
this training outcome was not reported In Good and Grouws' study. It Is a 
reasonable outcome to expect from this type of staff development program* 
Correlational results shown In table 4-14 indicate that there Is a moderate 
negative relationship between Implementation of the Instructional model and stu- 
dent off-task behavior. 

The results summarized above raise several questions. Why did the staff 
development program work? Why did It work for teachers whose principals were 
not Involved In the program (that Is, the regular inservlce group)? And why did 
It not work for all teachers? We shall discuss each of these questions In turn* 

Why did the program work? Good and Grouws' Active Mathematics Teaching 
worked, we believe, for two reasons. First, the design of the program includes 
a number of features that research has found lead to Implementation of the 
program's content. (Reviews of research on effective Inservlce practices are 
available 1n Gall and colleagues 1982, and Sparks 1983). For example, the 
objectives are clearly articulated In the form of a dally and weekly lesson 
plan; the sessions provide opportunity for expression of teachers' personal and 
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technical concerns; there Is opportunity for practice between sessions; the 
Instructional model Is generally consistent with teachers' belief systems about 
mathematics Instruction; and the program generally requires "fine-tuning" of 
existing skills rather than acquisition of a new skill repertoire. These 
features of the program were probably Instrumental In promoting the observed 
changes In teachers' Instructional behavior. 

The effects on student achievement and off-task behavior can be accounted 
for by another aspect of the program's design. The Instructional model taught 
In the program originated In correlational research on effective mathematics 
Instruction. (This research was discussed In Chapter 2). Thus, each element of 
the Instructional model was Included because of evidence that It promotes stu- 
dent achievement and on-task behavior. Indeed, as teachers gained experience 
with the model, they saw that It was having positive effects on students, and 
this perception encouraged further Implementation of the model. Teachers 
reported In the training and end-of-experlment sessions that students responded 
especially well to the predictable dally lesson plan, to mental computation, to 
homework, and to the fact that their work was checked. 

Why did the program work for the regular Inservlce group ? The principals of 
teachers In the regular Inservlce group did not have the same Instructional 
leadership training and support provided to the Involved principals. Yet the 
results Indicated that Good and Grouws' program worked almost as well with the 
regular Inservlce group as with the principal Involvement group. As we 
discussed earlier In the chapter, statistically significant differences between 
the two treatment groups were generally of small magnitude. 

One explanation for this result Is that principals In the regular Inservlce 
group also engaged In Instructional leadership behaviors. All of them were 
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experienced principals In their districts. Principals In the large district 
were using Madeline Hunter's Instructional Theory Into Practice model for super- 
vising teachers' Instructional behavior. Principals In the two small districts 
were using the Valley Education Consortium's model for assessing student 
achievement and for linking assessment to curriculum. 

The district-sponsored models, then, focussed principals' attention on aca- 
demic achievement goals, classroom Instruction, and teacher supervision. Thus, 
the difference In Instructional leadership between the two trained treatments 
probably was a matter of degree rather than of present-absent. 

It Is important to note In the preceding analysis that the Initiative for 
principals' Instructional leadership came from the school district central 
office. Participation of principals In the Valley Education Consortium or In 
Madeline Hunter's program represented district Initiatives. This fact, together 
with other features of the experiment, suggests to us that the leadership beha- 
vior of district administrators was an^ Important determinant of the effects 
observed In this study. Indeed, the experiment could not have been conducted 
without the Involvement of district administrators. The assistant superinten- 
dent In each district was contacted Initially to determine the possibility of 
having district schools participate In the experiment. These contacts were made 
by a member of the research staff who had a reputation for state-level 
leadership In school improvement, and who was a central figure In the development 
of the Valley Education Consortium. His authority as an expert and person of 
Influence was doubtless Instrumental In persuading the assistant superintendents 
to take a close look at this experimental project. 

The next step In the process was to meet with the principals In each 
district to secure their support. These meetings Involved all of the principals 



who eventually participated In all three treatment groups. Thus, principals In 
the regular Inservlce group were subjected to the same elements of persuasion as 
principals in the principal Involvement group. The principals then had the 
opportunity to meet with their teachers before making « final commitment to the 
project. 

This pattern of recruitment Is not unusual In field research In education. 

For example, Good and Grouws (1981) recruited participants for their junior high 

school experiment using this procedure: 

The Investigators met with school administrators during the 
summer In order to explain the project and to obtain permission 
to do the study. School administrators explained the project 
to principals, who In turn described the project to classroom 
teachers (p. 40). 

The fact that this Is a typical recruitment procedure makes It easy to overlook 
the role of school district administrators In research on classroom Instruction, 
This oversight Is also augmented by the fact that district administrators 
usually fade. Into the background once a project of this type gets underway. 

It appears, then, that principals In the regular inservlce group were sub- 
ject to persuasion by school district administrators and by the research staff 
to participate in the experiment. Also, after the experiment was approved, 
these principals knew that it had district support and sanction. Furthermore, 
teachers of these principals were given cues that the experiment had the support 
of district administrators behind it. The primary trainer in two of the 
districts was a central office supervisor; in the third district this person was 
a longtime teacher with a reputation as a district leader. Both principals and 
teachers in the regular inservlce group knew that data on instruction and beha- 
vior were being collected and would be reported in the form of summary sta- 
tistics to district administrators. 
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This analysis suggests that leadership behavior by district administrators 
had an Influence on the effectiveness of the regular Inservlce group. It is 
difficult to Imagine that a building principal and his or her teachers could 
Initiate or sustain a staff development program of this type without the 
involvement of district administrators. 

The Investigation of instructional leadership by district administrators was 
not an objective of this experiment. The results, however, suggest that It Is a 
distal, but nonetheless Important determinant of teachers' Instructional effec- 
tiveness. We suggest as a hypothesis for further Investigation that district 
administrators and building principals do not produce leadership effects Inde- 
pendently, but rather they work Interactively to produce them* Surprisingly, 
the research on school and teacher effectiveness has focused almost exclusively 
on the principal's instructional leadership. The analysis of present results 
suggests that a broader framework that Includes both district and principal 
leadership behavior Is needed. In the next section we propose some Initial ele- 
ments of this framework by Identifying leadership functions at different levels 
of administration that may affect school and teacher effectiveness. 

Why was the program not more effective? Although the trained groups were 
more effective than the control group on most outcome variables, the training 
effects were not exceptionally strong. For example, the trained groups used an 
average of eight elements of the Instructional model to the criterion level (see 
table 4-1) but the complete model Includes thirteen elements. Approximately 
50 percent of the trained teachers did not do mental computation exercises for 
at least three minutes In the posttralning lessons; about 60 percent did not do 
controlled practice for at least one minute; about 25 percent did not assign 
homework; and about 60 percent did not spend at least one minute checking seat- 
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work (see table 4-3). Almost 20 percent of students were off-task during mathe- 
matics Instruction, even after their teachers had participated In the staff 
development program (see table 4-11). 

Some of these Implementation failures can be explained by Individual dif- 
ferences among teachers. In a review of research on this problem, Mohlman, 
Coladarcl, and Gage (1982) found that teachers who perceive an Instructional 
practice as easily used and congruent with their Instructional approach are more 
likely to Implement the practice than teachers who perceive It as difficult and 
Incongruent. Similarly, Ebmeler and Good (1979) found that Implementation of 
Instructional practices In the fourth grade was partly a function of teacher 
characteristics. 

The finding that teacher characteristics affect program Implementation seems 
very reasonable. Further analysis, however, suggests that the relationship 
between these factors may be mediated by administrative policy. If teachers 
perceive that they have a choice about Implementing the practice, their 
personal characteristics will determine their Implementation behavior. If 
administrative constraints are placed on choice, however, personal charac- 
teristics are much less relevant. 

Our experience with the training sessions in the present experiment suggests 
that there was a delicate balance between administrative constraints and teacher 
choice. The Involved principals occasionally made recommendations, but they 
were careful to avoid directly requiring implementation of the model. The prin- 
cipals did make offers to help in implemintation, the most notable example being 
the sending of letters over their signature explaining that teachers would be 
Instituting homework. Several principals also assured teachers that they would 
help teachers handle parents who might object to having their children do 
homework. (This possibility did not materialize, to our knowledge). 



For the most part, consensus decision-making seemed to operate. Teachers 
generally agreed to try the model, but If a teacher vigorously objected to 
Implementation, no coercive behavior was used. In fact, only one teacher 
objected to the point that he wished not even to have his classroom observed at 
the time of the post and delayed lesson. He had the principal's support In this 
decision. 

This analysis Is consistent with other analyses that have been made of 
school organization and administration. For example, the concepts of school 
cellular organization (Lortle 1975) and loose coupling (Ueick 1976) acknowledge 
the relatively weak control of administrators over classroom Instruction. Their 
relevance to the present experiment Is that they place a limit on the extent to 
which principals can exert control over teachers' Implementation of practices 
contained in a staff development program. The leadership interventions used In 
the principal Involvement treatment (training session attendance; classroom 
observation and conference) may have a low upper limit of effectiveness. To 
Increase Implementation beyond a certain limit. It Is probably necessary to 
Institute mechanisms that place constraints over teacher choice. Otherwise, 
teacher characteristics will operate and determine whether the Instructional 
practice Is Implemented. Although these characteristics may be modifiable, the 
process Is likely to be time-consuming and expensive. 

Nonrandom assignment of principals to treatments. In discussing the dif- 
ferential effectiveness of the principal Involvement and regular Inservlce 
treatments, we noted the problems of Interpretation created by the nonrandom 
assignment of principals to treatment. The same problem affects comparisons 
Involving the trained groups and the control groups. Because of nonrandom 
assignment, there was a systematic bias In the type of principal selected for 
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the control group. As stated In Chapter 2, control group principals, even 
before this experiment, participated less often In teachers' Inservlce sessions 
than did principals In the two treatments Involving training. Also, control 
group principals In District 1 less often used Madeline Hunter's Instructional 
Theory Into Practice as their basis for teacher supervision. 

Despite the pre-existing treatment group differences on these dimensions, the 
control group was very similar to the trained groups on pre-treatment Instruc- 
tional variables relating to teacher behavior, student behavior, and student 
achievement. Pre-existing differences within and between treatment groups were 
controlled statistically for their effect on the dependent variables. Also, 
treatment group differences on Implementation variables were very similar to 
differences found In Good and Grouws' fourth-grade experiment (see table 4-13), 
where random assignment was present. 

Because of these factors It seems very unlikely that the superiority of the 
trained groups relative to the control group on the dependent variables was due 
to Initial superiority of trained group principals In Instructional leadership. 
The observed treatment group differences on the dependent variables can be 
attributed with a high level of certainty to the staff development program that 
was used In the principal Involvement and regular Inservlce treatments. 

The pre-existing differences between treatment group principals does pose 
problems for general Izablllty of findings, however. There Is little assurance 
that selection of an;^ group of principals for Instructional leadership training 
of the type operational Ized In the principal Involvement treatment would produce 
similar effects to those observed here. Rather, It seems likely that Instruc- 
tional leadership training would only be selectd and Implemented by principals 
who were predisposed to be instructional leaders. 
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Individual differences between principals In attitude toward Instructional 
leadership should come as no surprise to educators. Research on staff develop- 
ment for teachers has documented repeatedly that some teachers Implement 
training more than other teachers as a function of personal characteristics 
(Mohlman, Coladarcl, and Gage 1982), Principals should be no different In this 
respect. The present experiment was not designed to explore the effects of 
Individual differences In principal characteristics on treatment implementation, 
but this could be done In future research. Hall, Rutherford, Hord, and Huling 
(1984), for example, have Identified characteristics of principals that look 
like promising predictors of Implementation of Instructional Innovations. These 
characteristics could be Included as variables in research on principal Involve- 
ment In teachers' staff development. 

Instructional Leadership Functions 

The present experiment was designed to focus on one leadership role - the 
elementary school principal ship - and the leadership functions of this role that 
might facilitate staff development programs for teachers. We decided on this 
focus because of recent research Interest In building principals and their 
Influence on school effectiveness. The results of the experiment, discussed 
above, suggest that other educator ro'!es and other leadership functions may be 
equally Important In understanding school effectiveness. 

Gersten and his colleagues (1982) made a similar observation. They argued 
that It Is more productive to understand Instructional leadership by analyzing 
It into functions, or "critical behaviors," than by focusing on a single 
leadership role like the princlpalshlp. Following their argument, we did a 
retrospective analysis, based on the present experiment, of the leadership func- 
tions that facilitate the implementation and effectiveness of a staff develop- 
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inent program of the type developed by Good and Grouws. 

Leadership functions . The analysis yielded this set of leadership functions 
for facilitating staff development (and, more generally, school Improvement): 

1. Pr1or1ty«sett1ng function . It requires an act of leadership to propose 
priorities for school Improvement. Educators In each of the three par- 
ticipating districts had Identified improvement of mathematics Instruction and 
staff development as priorities for their districts. The salience of these 
priorities was used by district administrators as the rationale for involving 
their staff In the experiment. 

2. Resource acquisition function . Leaders In the two small districts 
chose to Involve their schools In the Valley Education Consortium over a period 
of several years because they viewed It as a continuing resource for school 
Improvement. The leaders In the large district made a major commitment to 
Instructional Theory Into Practice over a period of 10 years for the same 
reason. Leaders In each district viewed the present experiment as an oppor- 
tunity to strengthen existing resources at virtually no cost. (In fact, 
honoraria and no-cost training were provided to participating schools.) 

3. Monitoring functions . These leadership functions Include seeing that 
all elements of the school Improvement process occur as Intended* Timelines 
need to be established; people need to be contacted; meetings must be called; 
resources need to be secured; trouble-shooting must occur as problems arise. 
Unless someone oversees this process, it can break down. 

4. Compliance function . This is one of the most critical leadership func- 
tions, but it is not well articulated in the recent literature on instructional 
leadership. There appears to be an assumption that teachers will Implement an 
instructional practice if they are trained well, if their concerns are 
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addressed, and If they are Involved In decision-making. Our observations 
suggest that leaders can and do supplement these strategies by various behaviors 
that are Intended to bring about compliance with the recommended Instructional 
practice. Examples of these behaviors are use of rewards (Including encourage- 
ment and emotional support) for compliance and sanctions for noncompliance; 
persuasive argument, a quality that one researcher has labelled "communicative 
potency" (Robert Mattson, personal communication); appeal to a higher authority 
on the need for compliance; expressions of enthusiasm for the merits of the 
instructional practice; and direct orders to comply (not observed in the present 
experiment). 

5. Training function . A leader may do all or some of the training called 
for in a particular staff development program, including training to develop 
teachers' instructional skills. Training can involve such techniques as holding 
meetings, modelling the practice, and conducting classroom observations and con- 
ferences* 

6. Instructional policy-making function. This function surprisingly has 
been overlooked by researchers of instructional leadership. Certain instruc- 
tional practices involve norms, resources, and staff beyond the individual 
teacher and his or her classroom. For example, we discovered that a teacher 
feels uncomfortable about initiating homework unless his or her colleagues also 
do so. Two teachers at the same grade level may wish to teach their math lesson 
at the same time so they can group students by ability level. This practice 
involves resources (for example, a certain type of school schedule) and use of 
staff that require shifts in instructional policy. Leadership is required to 
recognize instructional policy issues and to make decisions about them. 

7. Assessment function. An important goal of school improvement efforts. 
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Including the present experiment, Is student achievement gain. Leadership Is 
needed to assure that appropriate measures of achievement are administered, 
reported, and used In decision-making. In many but not a11 respects, the 
assessment function overlaps the Instructional policy-making function. For 
example, assessment results can be reported as group averages (the usual proce- 
dure) or by Individual students or subgroups to focus attention on students with 
special Instructional needs. 

^* External relations function. Leadership Is required to maintain 
healthy relationships with Individuals and agencies that wish to affect or are 
affected by the school Improvement process. For example, assignment of homework 
was a critical element of the Improvement process In our experiment. This 
Instructional practice necessitated a change In school-home relationships, and 
leadership was required to Insure that the transition proceeded smoothly for all 
Involved. Also, the relationship between the Valley Education Consortium and 
the research staff for this experiment needed occasional attention to Insure 
that the work of each group was mutually beneficial. 

^* Maintenance function . One of the most critical leadership functions Is 
to establish mechanisms for Insuring that a school Improvement process Is main- 
tained over time. Substantial momentum was generated In the three districts to 
deliver Good and Grouws' staff development program to all fourth- and fifth- 
grade teachers In the schools assigned to the two training treatments. Once the 
experiment ended, the research staff largely withdrew from the districts. One 
staff member In each district was trained to deliver the program, but con- 
tinuing leadership will be required for these trainers to deliver the program to 
control group teachers and to new teachers entering the district. Also, policy 
decisions need to be made about delivering the program to teachers at other 
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These nine leadership functions are very briefly sketched In the above ana- 
lysis. The significance of the analysis for present purposes Is that It reveals 
the magnitude and complexity of Instructional leadership required to bring about 
school Improvement. Previous lists of leadership functions (for example, 
Sweeney 1982) are much narrower In scope, we believe, because they focus on a 
single leadership role — the building principal role — rather than on the 
range of external agency, district, and school roles required to launch. Imple- 
ment, and sustain a school Improvement process. 

The design of the principal Involvement treatment reflected this narrow 
scope because It was based on research on principal leadership. The treatment 
emphasized the principal leadership functions of attending teachers' staff deve- 
lopment sessions, reviewing student achievement scores, and conducting 
observation-conference cycles with teachers. It was the relative success of the 
regular Inservlce group that led us to view instructional leadership as a 
broader phenomenon and to analyze It into the nine functions described above. 

LeadersJilp roles . Some research on principals supports the Importance of 
their role as Instructional leaders, whereas other research questions the Impor- 
tance of that role (Gersten et al. 1982). From our current perspective, these 
disparate findings are not necessarily contradictory. Some of the nine 
leadership functions Identified above can be performed effectively by prin- 
cipals, or by other personnel. For example, district supervisors and curriculum 
specialists probably have the most expertise and most time to perform the 
training, assessment, and maintenance functions. The building principal may be 
the best qualified, by virtue of interest and designated role, to provide 
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leadership for the compliance, coordination, Instructional policy-making, and 
school-community relations functions. The superintendent and assistant superin- 
tendents, again by virtue of Interest and designated role, may be the most 
qualified to perform the priority-setting and resource acquisition functions. 
Teachers may be best qualified to provide leadership. In conjunction with admi- 
nistrators, for the training. Instructional policy-making, and compliance func- 
tions. 

We do not mean to Imply that unique leadership functions can be ascribed to 
each educator role In a district. The functions clearly overlap roles. The 
point of this brief analysis Is that the present experiment suggests the need to 
Include a variety of leadership roles and functions In planning a staff develop- 
ment process Intended for school Improvement. These roles and functions extend 
well beyond the school building level. Current opinion, however. Is that a 
building level focus Is necessary and sufficient. For example, Neale, Bailey, 
and Ross (1981) stated that staff development should be "located In the local 
school building, directed by the principal and staff of that building to meet 
educational needs Identified by the staff and clientele of that school." Our 
experience In the present study suggests that the principal and staff can pro- 
vide much leadership for staff development, but that additional leadership 
resources outside the school are needed too. 

The Relationship Between Staff Development, School Organization, 

and School Improvement 

It may seem presumptuous at first glance to generalize from a single study 
about the relationship between staff development programs and school organiza- 
tion. The attempt seems less presumptuous, though, when we pause to consider 
that the experiment extended over three different school districts, 53 teachers 




(35 of whom were specially trained), and 15 principals (five of whom were spe- 
cially trained). Also, the staff development program used In the study Is 
representative of a new generation of such programs In which Instructional beha- 
viors have been empirically linked to student achievement gains. (These 
programs are reviewed In Gage and Glaconia 1981). 

Our experience In this experiment suggests that the manner In which schools 
are presently organized creates obstacles to successfully Initiating, Imple- 
menting, and sustaining staff development programs of the type represented by 
Good and Grouws' Active Mathematics Teaching. Similarly, present school organi- 
zation acts as an Inhibitor to the performance of leadership functions required 
In using this type of program. 

Weick (1976) and other organizational theorists have characterized schools 
as loosely coupled systems. This term is used to characterize an organization 
In which elements of the system are attached, but the attachment Is 
"circumscribed. Infrequent, weak In Its mutual affects, unimportant, and/or slow 
to respond" (Weick 1976, p. 3). With this meaning In mind. Gall and his 
colleagues (1982) Interpreted current patterns of staff development as a loosely 
coupled system In school districts: "[Current staff development programs] con- 
sist predominantly of Isolated Inservlce activities focusing on teacher develop- 
ment rather than school Improvement. The activities are loosely connected to 
priority goals and to assessment of student outcomes" (p. 117). 

An analysis of Good and Grouws' staff development program Indicates that It 
implies a tightly coupled school system, or at least an effort to tighten the 
coupling of school elements. For example, the effects of the program can be 
directly related to district and school goals for Improvement. Teacher Instruc- 
tional behavior Is linked to student behavior In the classroom and subsequently 
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to student achievement. Each of these elements (teacher behavior, student beha- 
vior, and student ach1e'*«ment) can be measured, and so there Is the potential 
for administrative control over them that does not exist In staff development 
programs designed for a loosely coupled system. 

If traditional school organization and staff development form a loosely 
coupled system, the Imposition of a tightly coupled program of staff development 
seems bound to create substantial organizational tension. Our experience 
suggests that the tension results from the Interdependence and complexity of 
resources required for implementing a tightly coupled program. For example, many 
human resources must be redirected from other activities In order to get the 
program underway. A massive effort Is required to assess student achievement on 
curriculum-referenced tests over several points in time. (If this work hid not 
been previously done by the Valley Education Consortium, the assessment could 
not have been done within project resources.) The need for supervision of 
teachers' behavior as they learn Good and Grouws* Instructional model Is still 
uncertain, but If It Is necessary, substantial resources are needed. Also, 
substantial collaborative decision-making Is required to develop the Instruc- 
tional policies Implied by a tightly coupled model of Instruction. This colla- 
boration Is difficult because teachers usually work In Isolation from each other 
and from principals. Indeed, Tye and Tye (1984) recently argued that most 
school reforms will fall because "teachers are normally Isolated from one 
another... and... do not often come together In their schools to discuss currlcu- 
lar and instructional changes" (p. 319). 

The marshalling of resources for a tightly coupled system, then, is substan- 
tial. In the face of such requirements, it is easy to see that a loosely 
coupled organization would resist the system. The resistance would come, in 
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part, from district leaders who see that the requirements exceed the authority 
they traditionally have assumed within their districts and even the capabilities 
of their organization. 

This analysis of school organization, staff development, and school Improve- 
ment Is based primarily on this one project. Descriptions of other projects 
Suggest that similar conditions exist elsewhere. Levlne and Stark (1982), for 
example, conducted case studies of three districts that recently attempted to 
Improve student achievement In Inner-city schools. Their observations of the 
change process reveal that tp^sslve organizational and Instructional re- 
arrangements were necessary for this purpose. These re-arrangements Involved 
such elements as "years of staff development," "systematically coordinating 
federal, state, and local resour s for compensatory education," "reorganized 
teachers' resource centers so that all the materials available In the school 
were coded for essential skills," and taking "great pains to minimize teachers* 
recordkeeping" (pp. 42-43). 

If our analysis has merit. It suggests that there are severe limits on the 
potential of a building principal to Implement a tightly controlled Instruc- 
tional program. The resource requirements of the program exceed the designated 
leadership authority of most principals. The only alternative, then. Is to 
invoke leadership at higher levels of authority— leadership found at the 
district, state, and federal levels. Building principals cannot be Ignored, 
however, because as we found, they do make a difference In Implementation of a 
tightly coupled program once the necessary resources have been assembled and 
organized. 
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Recommendations for Future Research 

This research project provided a substantial experimental test of the propo- 
sition that school principals can be enlisted as a resource to enhance the 
effectiveness of a staff development program for teachers. This proposition, 
which Is grounded In previous descriptive and correlational research, was sup- 
ported by the results of the experiment. 

The experimental test was by no means conclusive, however. Additional 
research Is necessary to establish that the observed effects were not chance 
events or artifacts created by systematic bias In the formation of treatment 
groups. Tighter experimental control and a longer timeframe for tracing effects 
are obvious recommendations. In addition, we recommend that future research 
Include studies of the process by which the effects are produced. It Is not at 
all clear how principals work with teachers on a day-to-day basis to Influence 
classroom Instruction. How did principals In the principal Involvement treat- 
ment talk to teachers about the need for homework or for more whole-class devel- 
opmental Instruction? What did they tell teachers after observing their 
classrooms for Implementation of Good and Grouws' Instructional model? What 
compliance techniques. If any, did they use? Our research methdology did not 
enable us to answer these Important questions, but It Is certainly feasible to 
design studies that ' ild do this. 

We recommend, too, that Instructional leadership be reconceptuallzed In 
future research. Our notions about this concept were largely grounded In the 
research literature on elementary school principals. Our analysis of events and 
outcomes of the experiment suggests the Importance of a broader conceptual fra- 
mework. An expanded set of leadership functions and district-level roles needs 
to be considered. A broader framework will complicate the problem of designing 
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experiments on Instructional leadership, but we believe that It will also lead 
to the design of more effective Interventions than the one tested In this 
project. 

Lastly, we Issue a note of caution about research testing the generallzabl- 
llty of the findings of this project. The experiment was focused on elementary 
school principals. Intermediate grade teachers, mathematics Instruction, and a 
"tightly coupled" staff development program. The meaning of Instructional 
leadership and effective Instruction Is likely to shift substantially as one 
focuses on other levels of schooling, on other curriculum domains, and on other 
types of staff development. If this Is true, the general Izablllty of the 
observed treatment Is likely to be low. Rather than testing the limits of 
general Izablllty, we recommend repllcatlve studies of the same situation for the 
purpose of developing deeper understanding of Instructional leadership pheno- 
mena. When the operation of Instructional leadership In one situation is well 
understood. It should be possible to develop more powerful studies of related 
phenomena In other school contexts. 
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APPBMDIZ A 
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The letters A, B, and C are used to designate the three participating districts. The 
school ID'S are those used in the original coding of the data, and so are not in 
numerical order in this table. 



ERIC 



138 



144 



T—dwr ID Dlitrict ID School ID Grade ta 

Cootrol Group 



36 


A 


1 


4 M 


37 


A 


1 


4 F 


38 


A 


1 


5 F 


39 


A 


1 


5 M 


40 


A 


3 


4 M 


41 


A 


3 


4 M 


42 


A 


3 


5 F 


43 


A 


3 


5 F 


44 


A 


6 


4 F 


45 


A 


6 


5 F 


46 


B 


12 


4 M 


47 


B 


12 


4 F 


48 


B 


12 


5 M 


49 


B 


12 


4 F 


50 


C 


15 


5 M 


51 


C 


15 


5 F 




C 


15 


4 F 


53 


C 


15 


4 F 



SH468 



APPENDIX B 



Interview Schedule for Principals 



Date 



School District 



Principal 



Gathering Information about the Form and Level 
of a Principal's Involvement in Instruction-Related Activity 

Principals in all participating districts 
Interview Schedule 



Supervision 

1. Do you follow a general model or approach to supervision, e^., ITIP? If so, please 
describe the assumptions underlying the approach and the procedures you use to 
implement it. If not, describe the approach you take to supervision generally. 



2. During the last 10 days how many different classrooms have you observed in? 



About how long was the typicax observational period? 



What kinds of processes did the observations focus on? 



What did you do with information collected through the observations? 



3. During the year generally about how much time do you spend weekly observing 
teachers teach? 

4. Do you (or does the district) require teachers to submit instructional plans? 

Yes No 
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If yes check all that apply. 

long-range; i.e., yearly or semester plans 

unit plans 

weekly plans 

daily plans 



5. If teachers are required to submit plans, who reviews them? 
You Other (specify) 

. How do you review them? e.g., just check to make sure they're done? make 
comments on them? 



. Does a teacher's instructional plan influence what you observe during classroom 
visitations? If yes, indicate how. 



6. Do teachers submit to you on a regular basis information on their students' progress 
toward attaining desired learning goals? (Over and above the quarterly reports 
they prepare for parents) 

If yes, what form does this information take, e.g., test scores? teacher Judgments? 
observational records? 



Placement 

7. What role, if any, do you play in placing students in your building? 
Role in Grade or Classroom 

Placement Decisions Role in Promotion/Retention Decisions 



o 
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What factors do you consider when making grade or classroom placement decisions? 



What factors do you consider in making promotion/retention decisions? 



Program Monitoring and Evaluating 

8. What kind of evidence do you look to during the year as a sign that the math 
program in your building is producing desired effects, or is falling short of 
expectations, e^., test scores, teacher perceptions, parent comments, personal 
observations, etc.? 



9. What evidence, if any, do you look to at the end of the year, or over a series of 
years, to evaluate the effectiveness of the program? Please rank these factors 
according to their importance in determining overall effectiveness? 



10* In what ways, if any, do you share information about the effectiveness of the 
program with teachers? 



With central office staff? 



With parents? 



With local Board members? 
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Involvement with Teaeher Inserviee 

11. To what extent, if any, do you plan, schedule, or coordinate inserviee activities 
for teachers in your building? 



12« Do teachers in your building regularly exchange ideas and information with each 
other about instruction-related matters? 



If they do, please describe the form through which these exchanges take place and 
your perceptions of the kinds of outcomes produced by these exchanges. 



What role, if any, do you play in facilitating these exchanges? 



13. What incentives, if any, do you offer to teachers who stimulate and sustain their 
involvement in professional development? 



14. To what extent do you participate in teacher inserviee activities?. 



If you do participate, list those activities in which you have participated during 
1982-83. 
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What activities did you participate in last year? 



15. What is your overall impression of the quality and effects of the inservice 
opportunities provided teachers in your district? 



Managing the School 

as a Learning Environment 

16. Recent research has demonstrated a rather obvious fact: student absences contribute 
significantly to the problem of student low achievement. Do you or your staff 
use any particular procedures to control absence and tardy rates, e.g., hiring aides 
to call parents at night about absent or tardy students? 



17. Studies also have shown that students' learning suffers in classrooms that frequently 
are intruded upon, e.g., by requests for students to leave the room, by announcements 
on the intercom, by tardy students entering the room, etc. Does your school have 
any particular policies designed to minimize classroom intrusions? Please describe. 



18. As you well know, time allotted to instruction is associated with student learning. 
Does your school have any guidelines for teachers about the amount of time they 
should spend per week, or per day, on instruction in a particular subject area? If 
yes, please explain. 
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Questions about this Study 

19. Do you have any questions or concerns about this study? 
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May 1983 

MATH IM8BRVICB STUDY ^ 
Exit Interviews with Principals 

SehoQli with kiYolvad Prindpals 

Training SeMione 

1. Were there any particular positive effects or negative effects associated with your 
participation in the training sessions last February? 

Positive Effects Negative Effects 

for the principal for the principal 



for the participating teachers for the participating teachers 



2. What was your perception of the quality and appropriateness of the training sessions? 
of the teachers' handbook that supported the training effort? (ask if principal had 
opportunity to review the handbook outside the context of the training session) 



3. Are there any ways you can think of to improve the training program? 
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Prineipal's ObMrvation of 
TflMhan* Um of MoM 

4* Did you have an opportunity to make two formal observations of teachers^ use of 
the Good 6l Qrouws* model? Yes No 

If no, hov» many observations were made? 



5. What procedure did you use to conduct formal observations of teachers' use of the 
model? (e*g.» the checklist prepared by project staff, a self-made observational code) 



How did this procedure relate to the supervisory procedures you ordinarily use? 



6. What did you do with the information obtained through the observation? 



7. How do you feel teachers responded to the observations you conducted and the 
information coming from them? 
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8. How often did you make informal observations of teachers' math lessons? 
weekly 

about every other week 

about once a month 

once every couple of months 

once or twice between February and June 



9. If yoi' did make informal observations, did you learn different things from them 
from the formal observations? 



10. What would you say was the most difficult part of the Good &. Grouws' model for 
teachers to use? Why? 



What part w«»5 the easiest? 



What part, or combination of parts seemed to produce the largest positive effects 
for teachers and students? 
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Prineipal*! Role in Fostflring 
CoUoague Bzetenge 

11* Did you have an opportunity to arrange a meeting with participating teachers in 
your building to discuss and resolve any problems that developed about using the 
model? If so, what specifically was discussed at the meetings and what decisions 
were reached? 



If not, was there any other way in which you promoted teacher interaction around 
issues related to the Good & Grouws* model? 



Review of VBC Test OaU 
witli Teaelian 

12. Did you have an opportunity to review with 4th & 5th grade teachers results from 
the VEC mid<yeai' test? If so. what procedures did you use to review the test 
information? 



13. How did teachers respond to the VEC test data? 
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14* What is your impression of the quality and utility of the VEC test data? 



Support in Carrying Out 
Role of Involvad Prineipal 

15. Do you feel you received enough guidance and support to carry out your role In 
the inservice program? 



16, What additional or different forms of guidance and support would have been of help? 



Perceptions of Role 
as an ftivolved Prineipnl 

17. Were the demands of time and energy made by the project manageable or 
unmanageable for you? 
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18. Was there anything you did in your role as an involved principal that seemed 
particularly productive, either for you or your staff? 



Was there anything that seemed particularly unproductive or disappointing? 



Future PoMibiUti 



19. Would you recommend that other teachers in your building receive training next 
year in the Good & Grouws' model? 



20. Do you think the kind and level of involvement that you had in this project would 
be appropriate for other staff development efforts in your building or district? 
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May 1983 ■ ^ 

MATH INSERVICE STUDY 
Exit Interviews With Principals 

3elioQli Raealving TmOm liMnriee Without Prineipil bf olv«iflBt 



1. What reactions do you have, if any, to the Good and Grouws' approach to math 
teaching that 4th and 5th grade teachers in your building receiv ' training in this 
year? 

Possible Probes - In your view, is there anything particularly distinctive about this 
approach to math? If so, what leads you to think this? What's your sense of the 
teachers' response to the model? Did it help or hinder them in any particular 
way? What's your sense of the impact or potential impact, of the model on student 
learning? 



2. What impressions do you have of the quality of the training sessions for teachers 
that took place last February? What's the basis of your impressions? 



3. How did teachers react to the classroom observations that were made by project 
staff? 
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4. During the time the project was being carried out, February 1 to the end of the 
year, how many formal observations of each 4th & 5th grade teachers* math lessons 
did you make? (formal observation = observation has preestablished focus; 
observation extends full lesson; information from observation discussed with teacher) 



About how often did you make informal observations of these teachers' math lessons? 
weekly 

about every other week 

about once a month 

once every couple of months 

once or twice between February and June 



5. Based on the information available to you, what would you say was the most 
difficult part of the Good & Qrouws' math model for teachers to use? Why? 



What was the easiest? 



Which part, or combination of parts, if any, seemed to produce the largest positive 
effects for teachers and students? 



6. Have you done anyt'iing different in relation to the math program in your school, or 
in relation to observations of teachers' math lessons, during the last semester? If 
so, are these differences linked to the math inservice project in any way? 
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7. What, If wythlng , hiv» you hMrd about tht work done by the princlpaKs) in your 
distriet who twve taken an aetive part in the math intervloe program? 



8. Would you recommend that other teachers in your building receive training next 
year in the Good & Grouws' model? 



9. If similar inservice training was to be provided next year, would you consider it 
worthwhile to attend the training sessions? 



Suppleaentary Section on VBC Test Data 

10. Did you have an opportunity to review with 4th & 5th grade teachers' results from 
the VEC mid-year tests? If so, what procedures did you use to review the test 
information? 



11. How did teachers respond to the VEC test data? 



12. What is your impression of the quality and utility of the VEC test data? 
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MATH WSERVICE STUDY 
Exit Interviews With Principals 

CmtrU Sehooli 



What, if anything, have you heard about the math inservice project that some of 
the other elementary schools in the district have participated in this year? From 
whom did you get this information? 

Possible Probes - What have you heard about: a) the Good 6l Grouws* model? b) 
the training sessions for teachers that took place last February? c) the observations 
that were made of 4th & 5th grade teachers' math lessons? d) what the principals 
who were involved in the staff development did, and how they felt abut what they 
did? 

Veard from teachers Heard from teachers 

in your building who received training 



Heard from principals or other administrators in the district 



If principals have heard something about the project, ask this: 

Based on what you've heard, would you like your teachers to receive training in 
the Good & Grouws* model next year? If so, what do you think the main benefits 
would be? 
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3. During the time the project was being carried out, February 1 to the end of the 
year, how many formal observations of each 4th & Sih grade teachers' math lessons 
did you make? (formal observation » observation has preestablished focus; 
observation extends full lesson; information from observation discussed with teacher) 



About how often did you make informal observations of these teachers' math lessons? 
weekly 

about every other week 

about once a month 

once every couple of months 

once or twice between February and June 



4. Do you feel teachers in your building have done anything different in math this 
last semester from what they have done in the past? If so, are these differences 
linked to the math inserviee project in any way? 



5. Have you done anything different in relation to the math program in your school, or 
in relation to observations of teachers' math lessons, during the last semester? If 
so, are these differences linked to the math inserviee project in any way? 



6. If similar inserviee training was to be provided next year, would you consider it 
worthwhile to attend the training sessions? 
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APPENDIX C 

An Introduction to the Valley Education Consortium 
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APPENDIX C 

An Introduction to the 
VALLEY EDUCATION CONSORTIUM 



WHAT IS THE. VALLEY EDUCATION CONSORTIUM (VEO? 

It is people from many realms of education, including Ldaching, management, 
service, teacher preparation, and research, working together to solve educational 
problems of common concern. 

HOW DOES IT WORK? 

In addition to a formal organizational structure of a Board (the chief administrator 
from each member agency) and a Council (a key middle-management representative 
from each agency), the Consortium accomplishes its work through cross-strict 
work groups, seminars, and work sessions, school-based support teams, and special 
district projects. 

WHO BELONGS TO THE CONSORTIUM? 

Currently active members of the Consortium include: the Carlton Elementary, 
Cascade, Central, Dallas, Dayton, McMinnviUe, Salem, Silverton Union High, 
WiUamina, and Woodburn School Districts; the Marion, Polk, and Yamhill Education 
Service Districts; Western Oregon State College, and the Teaching Research 
Division of the Oregon State System of Higher Education. By virtue of their 
ESD*s involvement, all school districts in Marion, Polk, and Yamhill Counties are 
associate members. 

WHY DID IT BEGIN AND WHERE IS IT GOING? 

The Consortium began operating on an informal basis in the early 1970*s to help 
Western Oregon State College (formerly Oregon College of Education) develop 
its teacher preparation program. During 1974-75 the Consortium grew in 
membership as districts and agencies sought common ways of responding to the 
State Department's new Minimum Standards for Elementary and Secondary Schools 
in Oregon. In 1976 a formal charter for the Consortium was adopted, since 
that time VEC.has focused its efforts on designing a comprehensive approach to 
the ifflproveihent of school programs (using O^'egon's Standards for Elementary 
and Secondary Schools as a base) and on providing support to schoob involved 
in program improvement. Work thus far has centered largely on the besic skills, 
but it is now expanding to include more sophisticated subject areas, including 
the development of critical thinking and problem solving (reasoning) skills. The 
Consortium also is working actively with faculty and administrators from WOSC 
to develop programs that will prepare teachers to function effectively within 
Standards-based instructional programs. 
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APPENDIX D 

Trainer Guidelines for 
Good and Grouws' Staff Development Program 
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First Training Session 
Good and Grouws' Inservlce Program 

Objectives; 

1. To develop teacher enthusiasm for the program. 

2. To provide a rationale for why the program works. 

3. To help teachers Implement the program In their cidssroom. 

4. To help teachers learn how to write math lesson plans based 
on the program. 



Why the District Supports the Program: 

1. Relationship of the program to district goals and philosophy. 

2. The program Is research-validated. Research demonstrates that 
It helps teachers and students. 

3. The program Is based on what effective teachers do. ("Effective" 
teachers are those whose kids learn math well.) The teaching 
model builds on what teachers already do. Some teachers follow 
the model closely even without special training. 

4. The teaching model does not mean more work for the teacher. Also, 
It can be used with any curriculum. 



Research Project Associated With the Program; 

1. Some district teachers are not getting the training now. Some 
of you are getting It* but not your principal. Some of you are 
getting It along with the principal. 

2. It's Important not to talk to teachers or principals In other 
schools so we can find out which Inservlce approach Is best. 
We will share the results with you at the end of the test. 

3. Any questions about the classroom observations that have occurred 
over the past two weeks? 

4. Reassure everyone that all of the research data are confidential. 
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ASCD Videotape: 

1. Videotape 1s approximately one-half hour. 

2. Answer questions about the tape afterwards. 



Overview of the Teaching Model: 

1. Major components of the model 

a. dally review 

b. lesson development 

c. seatwork 

d. homework 

e. special reviews 

2. The essence of the model Is an Integrated sequence of: devel opment-prac- 
tice- review. 

3. Overhead transparencies can be used to present the overview. 

4. Answer questions about the overview. 



Distribute Teacher Handbook: 

1. As time permits, have teachers and principals read several sections 
of the handbook. 

2. Answer questions. 

3. Assignment: read rest of handbook by next training session. 

Construct a Lesson Plan Using the Teaching Model 

1. Distribute packet of lesson plan forms. 

2. Have teachers and principals construct a ma'ch lesson plan using 
the form provided. 
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Small Group Discussion; 



1. Give teachers and principals a chance to form small groups 

and discuss their reactions to the teaching model. Teachers whose 
principals are Involved should form separate groups school by 
school . 

2. Allow approximately 20-30 minutes for this activity. 



Concluding Remarks: 

1. Repeat assignment: read rest of teacher handbook. 

2. Start Implementing the model to the extent possible. 

3. Date of next session. 

4. Purpose of next session Is to discuss your experiences In Implementing 
the model , and to share Ideas about how to make the model work. 
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Whole Group versus Individualized Instruction 



Problems with Totally or Largely Individualized Math: 

1. Students tend to qet off task easily and stay off task for long 
periods of time. 

2. Children tend to get frustrated because It Is not easy to get 
teacher assistance. 

3. Many students at fcurth and fifth grade level aren't able to work 
Independently under Individualized Instruction. 

4. Not enough opportunity for teachers to explain concepts and 
procedures In detail. 



Key Points: 

1. Good and Grouws' teaching model Is not opposed to Individualized 
Instruction. Seatwork and homework can be Individualized. The 

model Just suggests doing certain activities (review, mental computation* 
development) with the whole group. 

2. Research shows that children benefit by spending as much time 

as possible with the teacher. In Individualized Instruction each 
child spends very little time with the teacher. 

3. The teacher Is critical In elementary school because many children 
don't have the reading skills necessary to learn from math textbooks 
and learning packets. Effective whole group Instruction maximizes 
children's exposure to the teacher. 
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Stcond Training Session 

Good and Grouvs* Inaervlce Program 



Introduction 



1. 
3. 

5. 



/ 

Answer questions about points made in the teachers handbook. 

Have teachers share experiences in implementing the teaching model* 

Show second part of videotape (complete math lesson illustrating 
the teaching model) • 

Answer questions about the videotape* 



Teaching Model 

Review each part of the model* and bring out the following points* 

DAILY REVIEW* The review should cover the previous day*s work* 
It needn*t take but a few minutes* The review helps children 
stay on track and reinforces what they have Just learned* 

COLLECT HOMEIiK)RK* Homework can be checked quickly by putting 
answers on the blackboard or on transparencies* Children can 
check their own answers* Spot-check a few problems to see If 
they*ve been done correctly* 

MENTAL COMPUTATION* It*s not important when in the lesson this 
strategy occurs • Children should do the computations in their 
head* Tliat is the point of mental computation* 

DEVELOPMENT* Teach for meaning* Teach not just how to do the 
computations » but w hy * Stress real-*life applications of the 
computations* Use manipulatives to develop concepts; manipulatives 
are quite appropriate for fourth and fifth graders* Try spending 
more time on development: children will retain what they We 
learned better and their seatwork and homework will be more 
successful* Let children reason the process through* 

CONTROLLED PRACTICE* Make sure you check for understanding 
before assigning seatwork* This is done by assigning a few 
problems similar to those included in the seatwork* Uave the whole 
class do each problem^ and check on the accuracy of their response* 
If students show lack of understandings engage in further 
development (step 5) * 
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7. SEATWORK. Try to assign enough seatwork so that students are 
busy the whole time. Monitor seatwork continuously rather than 
working on other activities. 

8. HOMEWORK. Ten to fifteen minutes Is sufficient. The purpose 
of homework Is to provide successful , distributed practice. 
Homework should reinforce the day's development and seatwork. 

9. SPECIAL REVIEWS. 
10. OFF TASK BEHAVIOR. 

a. mlnlmlie transitions from one activity to another. 

b. do not allow unlimited use of the bathroom. 

c. do not Interrupt the math lesson with other activities, e.g., 
distributing lunch tickets. 

d. avoid excesislve student movement In the room or leaving the 
room while the math lesson Is In progress. 

e. do not Interrupt children during seatwork. 

f. If a child is at the blackboard, keep other children Involved. 



Construct a Lesson Plan Using the Teaching Model 

1. Ask teachers whether they've used the lesson plan forms provided 
1n the first session. 

2. Have teachers and principals construct a math lesson plan using 
the form provided. 



Whole Group and Individualized Instruction 

1. Discuss problems In adapting the teaching model to Individual 
differences among children In learning math. 



Small Group Discussion 

1. Provide opportunities for discussion among participants. Teachers 
whose principals are Involved should form separate groups school 
by school . 

2. Allow approximately 20-30 minutes for this activity. 
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Conclusion 
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1. Remind participants not to talk with teachers and principals In 
other schools about this Inservlce program. 

2. Ask participants not to mention to observers which experimental 
group they're In. 

3. Give approximate dates of two classroom observations and student 
testing. 

4. Get teachers' math lesson schedules. 

5. Have teachers and principals complete end-of- session form. 
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Questions and Answers About the Teaching Model 



Q: How closely must we stick to the time allocations? 

R: We are asking that you stay reasonably close to the suggested time lines. 
Previous research suggests that these requests will result In higher student 
achievement and we ere anxious to give this Idea a rigorous test. Having 
said this, we realize that from time to time some variance will be necessary; 
we ask only that you give your best effort. 



Q: What about a student who often does not do the homework assignments? 

R: It 1s Important that students do the homework assignments. Try to determine 
why the assignments are not being done... too difficult? laziness? and 
act accordingly. Sometimes a talk with the student will help. Other 
times, a chat with the parents may be appropriate. We leave It to your 
professional judgment as to what Is best In your particular situation. 



Q: How can I Increase the amount of time I spend on development? 

R: First you mlqht consider whether you are spending appropriate amounts 

of time on Initial development of meaning, assessment, controlled practice, 
and reteachlng. Beyond this. Initial development can be profitably expanded 
by using new or additional concrete materials, or providing supplemental 
activities (that fit with the objectives of the day) generated from game 
books, other textbooks, professional Journals, or from colleagues. Remember 
that presenting Ideas 1n several different contexts Is part of development 
and often helps students to transfer the Ideas to new situations. 



Q: Have you got some additional Ideas for mental computation? 

R: You might work on multiplication and division by ten. Maybe we could 
share some Ideas on this with one another. Mrs. Wilson what have you... 
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Q: What about grading practices with this program. Does the homework count 
as part of the grade? 

R: We ask that you evaluate student performance using the standards you have 
always used. If you feel seatwork and homework performance should be 
a part of this assessment that certainly would be okay. In fact, that 
seems to be a very reasonable sort of thing to do. What part of the grade 
1s again left to your discretion. 
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Q: I have always used periodic quizes when I teach math. Can I use them 
with this program? 

R: There would be no problem with continuing to use these quizes. Of course 
using them on a dally basis would tend to Interfere with the Instructional 
pattern that we are asking you to follow. 



Q: Is It required that the reviews be done on Monday? 

R: We would like you to do them as often as possible on Monday so that we 
can get In the reviews at approximately equal Intervals. However, If 
things work out so that you can finish a unit or chapter on a Monday then 
by all means put off the large review until Tuesday and If you wish to 
give a unit test give It on Wednesday. 



Q: What about the frequency with which I test my students? 
R: This Is up to you and your Judgement. 
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APPENDIX E 
NOTES ON TRAINING SESSIONS 



1. 
2. 



3. 
4. 
5. 

6. 
7. 
8. 

9. 



10. 
11. 



12. 



13. 



First Training Session In District 1 - January 31. 1983 

24 teachers and principals were In attendance. 
Trainers put up two posters. Poster 1: 
Schedule 

57170 - 8:15 

8:15 - 8:45 

8:45 - 9:30 

9:30 - 9:45 

9:45 - 10:30 

10:30 - 11:00 

11:00 - 11:30 

Poster 2: 

Objectives 
Participants will: 

- have an awar * 

- know and expla\:. 



Coffee 
Introduction 
Overview 
Break 

Good & Grouws model 
Lesson Planning 
Summary & Next Steps 



cf :he Good & Grouws model 
,he components and the lesson plan 
- design a lesson using the lesson plan 
Teacher asked, should mental computation relate to the current lesson? 
Teacher said, the model says nothing about testing and quizzing. 
Teacher said, the model does not talk about what to do with kids who 
finish seatwork early. 

Teacher asked, should you assign homework If students do seatwork wrong? 
Teacher asked, how should you check homework? Should you spotcheck? 
Teacher asked, how do you deal with different ability groups In the 
classroom? 

Trainer related Good and Grouws' concept of development to ITIP concepts 
of: "teaching to an objective," "monitor and adjust," "applied practice," 
"modelling," and "anticipatory set." 

Trainer had participants read selected pages from teacher handbook, 
especially section on homework. 

Teacher said, students shouldn't have homework. Students have enough work 
to do In seatwork. Parents can foul students up by teaching things wrong. 
Research staff members responded, research shows homework Is effective. 
Teacher said, his daughter, who Is In a second grade private school, has 
homework every night. He thinks homework Is good and assigns It. He 
checks homework when students start the math lesson. If a child hasn't 
done his homework, he has to do It during recess. This Is very effective. 
Several teachers said, they teach small groups during math, and this model 
of whole-group teaching will be a problem for them. 



First Training Session In District 2 - February 1, 1983 

1. Nine teachers and one principal were In attendance. 

2. Teachers said. If children get homework In every subject, that's 2 or 3 
hours of homework a night. This Is a concern of the teachers. 

3. Teacher asked, what do you do if you have a "b' ' "*9r (a preset-vice 
teacher doing a practicum) or student teacher "room? 
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4. Teacher said, Japanese students do very well, and they have total group 
Instruction and lots of homework. 

5. Principal said, don't spend so much time worrying about the bottom ten 
percent of the class. Work with the children who are ready to work. (This 
was said after some of the teachers said that slow students In the class 
wouldn't do homework or get on-task quickly as lesson shifted from one 
activity to another.) 



First Training Session In District 3 - February 3. 1983 

1. Session started with assistant superintendent giving a pep talk on the 
district's support of the project. He left after about ten minutes. 

2. Teacher asked, how does one form classroom groups to handle Individual 
differences? Another teacher suggested. If you have two fourth-grade 
classrooms, you can form two groups. One teacher can take the high group, 
and the other teacher can take the low group. 

3. Teacher complained, "We don't have a choice. District wants us to 
Implement the program." Teacher referred to the assistant superintendent's 
pep talk at the beginning of the session. 

4. Teachers recommended, district should send the letter to parents 
concerning homework. (Good and Grouws' teachers manual contains a sample 
letter than can be sent to parents explaining the shift to use of homework 
assignments.) Much conversation about this procedure. 

5. Teachers talked a lot about slow learners In their classroom. 



Second Training Session In District 1 - February 17, 1983 

1. A schedule was presented on a poster: 

8:15 - 8:30 Set 

8:30 - 9:00 Review 

9:00 - 1C:00 Problem-Solving 

10:00 - 10:15 Break 

10:15 - 10:30 Observation Form 

10:30 - 10:45 Test Results 

10:46 - Mils School Groups 

11:15 - 1*00 Summary 

2. Another poster stated objectives: 

Participants will: 

- refine their understanding of the Good and Grouws lesson design 

- review solutions for classroom Implementation and problems 

- explain how to read the observation form 

- understand the VEC test printout. 

3. Trainer made constant reference to the relationship between Good and 
Grouws' model Madeline Hunter's model. 

4. Teacher asked, suppose children don't get the concept during seatwork. 
Should you assign homework? Trainer responded. Yes, to maintain the 
expectation that there will be homework. 

5. Teacher asked, how do you get sufficient time to reteach the small group 
of children who aren't getting It. 
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6. Teachers expressed tension about doing a review and finding out children 
didn't get It. Do you march on to new concepts or reteach It? 

7. A few teachers said they have highly Individualized programs and would find 
It very difficult to switch to the whole group approach recommended In 
Good and Grouws' model. 

8. Teachers said, there Is a great range of SES differences In their district. 



Second Training Session In District 2 - February 16, 1983 

1. An account of this session by a CEPM staff person not affiliated with this 
research project Is presented at the end of this appendix. 

2. Eight teachers, the principal, and a block student were present. 

3. A teacher complained about two very slow learners In her group. They're 
just not learning math In his classroom. 

4. Teacher said. It's good to collect homework at start of the day so children 
don't do homework In morning before math period. 

5. Teacher said. Good and Grouws model Is good because low ability students 
find security In Its structure and routine. 

6. Teacher observed, students are making good progress after she started 
giving homework consistently. 

7. Teacher said, she really likes mental computation part of the lesson. 

8. Principal asked, do teachers use manlpulatlves In their math lessons. 
Teachers said there was a lack of manlpulatlves for them to use. 

9. Teachers said, they feel bound by having to march children through the 
curriculum. 

10. Teacher said, one parent wrote her a letter about the new homework policy, 
"It's about time." 

Ur Teacher told the principal, tall office not to call me over the Intercom 
during math period. 

Second Training Session In District 3 - February 18> 1983 

1. Teachers said, children like a predictable structure. They like knowing 
there will be mental computation each day. 

2. Teachers spent a lot of time talking about the few children who are way 
behind In math. 

3. Teacher said, there Is a benefit to multiple demonstrations. More and more 
children tune In with each demonstration. The light bulb goes on for them. 

4. Some teachers report, they are doing switching. Slow children go to one 
teacher; bright children go to the other teacher. 

5* Twelve participants attended today: one principal, 9 teachers, and 2 
student teachers. 

6. Teacher said, math Is very abstract. Students need concrete manlpulatlves. 
(This remark and similar remarks In the other districts were -ompted by 

the extensive use of manlpulatlves In the videotape shown In .ond session.) 

7. Teacher said, children feel It's Important to finish their seatwork, even 
at the expense of accuracy. If seatwork Is too long, children tend to get 
off task. It Is helpful to use some children as tutors. 

8. Teacher said, she assigned this homework problem - make up a story problem 
for the other children to solve. 

9. Several teachers said, they really get bogged down with hard-to-teach kids. 

10. Teacher asked, what is the best time of day to teach math? 

11. Teacher said, spending more time on math takes away from other subjects. 
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Wddneaday Morning 
Live: ObMrvations 
on a Staff 

Development Meeting 

ByWynnD«B«volM 

The boardroom in thto elementary- 
3chool4umed-dislrfct-office looks out 
on a large warehouse. Almoet loit 
against the structure's gray weitfiered 
shingles, pigeons gather and dlii>erse. 
their meetings brief and fiduL 

inside the boardroom, elementary 
teachers talce their seats slowly and 
deliberatefy. It is eight-fifteen in the 
moming and they are waiting for a 
training session to begin. As the uni- 
versity professor and the district trainer 
prepare the audiovisuals, the teachers 
joke arKi (H'.umble about being there. 
They regard the questioruMlres they 
have been asked to fill out uith evident 
distaste. Some push them away. Their 
faces resemble those of students who 
feel they've been given an unfair 
homework assignment The professor, 
a consultant for 1^ staff devek)pment 
program, mild^^ ' explains the impor- 
tance of their ananwers. He does not 
react to the restrained hostility. As the 
meeting b ready to begin, most of the 
teachers Jtmp up to get a cup of 
coffee from the kitchen down tSe hail. 
One laughs and observes that their 
behavk>r is "off-task." 

By the time everyone returns, the 
principal has arrived and the atmos- 
phere Is more subdued. Some are 
now filling out the questionnaires that 
ask for their reactions to the first two 
weeks of using a new jnstructkmai 
method in their math ktssons. 

For the next 50 minutes, all partici- 
pants view a videotape of a teocher 
using Thomas L Good and Douglas 
A. Trouws' Active Teaching of 



Mathematics system in a fourth-grade 
classroom. It is Cood and Qrouws' 
model that the teachers gatiwred in 
this room have been fol^ving for the 
past two weeks. As they watch the 
vkleotape. several teachers continue 
to fill out the questionnaire. They are 
intent and serious as they write, but 
they also manage to fottow the taped 
presentation. The principal's attention 
. does not waver from the videotape. He 
seems to be digesting every activity 
and interaction In the filmed lessoa 

In the eNchange of views ttiat follows, 
the teachers bring up problems they 
encounter in class that are not ad- 
dressed in tfie vkleot^>e-disclpllne, 
length of time it takes for students to 
correct each other's papers, difficulties 
in planning lessons, and grouping 
problems. They complain that 
"canned" presentations never quite 
reflect tiieir own unique classrooms. 
As each teacher mentions a problem, 
several otiiers offier suggestions and 
technk|ues that work for ttiem. 

Initially, ttie interaction takes place 
among the teachers without interrup- 
tion by the trainer, principal, or profes- 
sor. Increasingly, however, questkms 
are posed directly to the trainer or the 
professor, and the principal asks 
questions witti greater freedom. He 
sits between the trainer and the profes- 
sor, opposite most of the teachers. 
*7he amount of time on task in the 
videotape was impi«ssive," he remarics. 
The comment hangs motionless for a 
moment, but no one suggests that the 
vkleotape represents an unachievable 
kleaL 

Discujfcikxi returns several times to 
issues that are dearly of great cortcem 
to the teachers. Ore such issue b 
homework. Before the district agreed 
to participate in the present experiment, 
teachers adhered to a policy of not 
assigning homework. Cinder the Good 
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and Qrouws system, they are expected 
to assign homewofk da^. In the pre^ 
ceding meeting^ there was resistance 
to this new policy. Teachers predicted 
that half the students would not do the 
worlt and that parents would complairu 
The principal inteivened althls point 
and reiterated Good and Qrouws* 
specifications for homework assign- 
ments: they should be given daily, 
require no more than 15 minutes to 
complete, and reinforce skills the 
students have already acquired In 
additkMi. he signified that the teachers 
shoukl feel positive about the 50 
p<ercent who do complete the assigned 
work. *View the cup as half fuU," he 
remarked, '*rather than as half empty." 

Apparently the teachers have taken 
the principars point to heart and have 
gained a new perspective on the issue. 
Now they are not discussing whether 
they should give assignments to be 
completed outsfcie of dass, but how 
they can most efficiently handle the 
whole process. One young woman 
wonders when to collect homework. "If 
•homework is to be reviewed the first 
thing during math class," she says, 
"and math is a fifth-period dass, do we 
collect homework at the beginning of 
the school day or after the work has 
been reviewed?" An older nnale teacher, 
obvfcNisly a veteran of many years, 
points out that if students are expected 
to complete their homework outskle 
of class, it has to be collected first 
thing in the moming. Another rruin, 
younger and not afrakl to show en- 
:iiusiasm, suggests making a transitkxi 
from the preceding dass to math by 
passing back the homework that had 
been collected at the beginning of the 
day. "When possibte," he adds, "I have 
already corrected the work by tlie time 
1 give it back for revkw. Of course, 
that's not always possible." 
The teachers' comments indkate 



that left to themselves they might not 
have devised a consistent method for 
handling practkre work and grading. 
The necessity imposed by the new 
InstructkMwl method, the guidance 
provided by the training program, and 
the principal's reassurance give them 
a sense of security in attempting new 
activities. With this support all appear 
to have established organized ways of 
coping with the assignment and grad- 
ing of homework. 

Tm getting a positive response." 
remarks one teacher. "There's cooper- 
ation from home; everyone's expecting 
homework." She seems slightly sur- 
prised by her success. "In fact, I think 
the kkjs like the structure of regular 
homework. They like to know wiiat's 
coming. And they're eager to see how 
they've done on last night's assign- 
ment" 

"Yes," adds another. **And now 
when a student or the parents come in 
and demand to know why 1 gave a low 
grade, 1 don't have to worry. I just open 
my gradebook and it's all right there." 

Another issue Important to the 
teachers Is that of student grouping. 
Befote the experiment, many had 
used some form of IndMdualixed 
instnjction. Qood and Qrouws' In- 
stnjctkXMl system emphasizes whole 
group in8tructk)n. All teachers express 
concern atxHJt how to handle classes 
in whkh most of the students catch on 
to new concepts qukMy but a few lag 
behind Do you aim for mastery by all 
students, sbwing down the pace and 
risking k>ss of interest in the brightest 
group? Or do you qukken the pace 
and accept the fact that the stowest 
students will not fiilly grasp nrK>st con- 
cepts? 

use students who 
UTKlerstand a lesson to hdp those who 
do not understand, '{"he professor 
assures them that the model is flexible 
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and eaiily aOowt for thto type of adap- 
tation. He adds other suggestions to 
help the teachers cope with students 
who learn at different speeds. "Dqplain 
the material in a new «vay for the stu- 
dents who didnt understand the initial 
explanatioa Sharir^ a different way to 
approach a problem deepens every- 
one's understanding. Or give students 
different ats/CMOik and homewor.k 
a9«ignnwits, depending on their abili- 
' tieSi 

Several teachers nod. The question 
comes upagaln. "Does the system 
alk>w for Individualization of activities 
during the tbne devoted to aeatworic?" 

"^^bsolutelyr responds the trainer. 
"Look, this system to designed to help 
you. We're not trying to chenge what 
you were already doing, we're trying to 
enhance it" Relief to expressed in 
smiles around the table. 

The prirKipal, wtio has been rein- 
forcing teadiers' suggestions, be- 
comes more actively involved in the 
dtocussion at thto point He wants to 
know how often they use a cfiscovery 
activity similar to the one demon^rated 
in the videotape. Their responses 
seem honest, not bound tiy what they 
thinic he would like to hear. Wouki it 
help, he wonders, if teachers gave the 
"greyhounds" creative work in class 
rather than homework. In untoon. the 
teachers say no. That type of differenti- 
ation wouki reward the bright students 
and it to the skiw ones that need to be 
rewarded He then questkxis them 
about manipulatives. "What manipula- 
tives do you have," he asks, "and what 
ones wouki you like to have?" 

One woman answers that she has 
some geo boards, which she uses for 
teaching about fractkHU. 

"Where dkl you get them?" asks the 

prindpaL 

"I rnade them." she responds and 
describes the materials needed 



Qearly. the principal to becoming 
famlltor wHh the smaU detaito of class- 
room nuMrMgement UTKler the Qood 
and Grouws system. 

At the conduston of this meeting, 
whkrh Is the last training session, 
teachers surprise the trainer by asking 
about getting together again at the 
end of the school year to review prog- 
ress. Their desire to devote further 
time to dbcusskxi indteates that they 
are finding the training benefkHal and 
belies the rehictant atmosphere with 
whk:h the sesston begaa 

These teachers do not feel they are 
voluntaiy participants in the im- 
plementation of the Cood and Grouws 
system. The project was approved and 
mandatedbydistfictadmintotrators.it 
to not surprising, then, that the partici- 
pants might feel misgivings and lome 
resentment about the training. What 
has become apparent however, as the 
teachers discuss problems and share 
solutkms related to using the model to 
their increasing desire to see the effects 
of its use on the VM>rk habits and skiito 
of their students. The encouragement 
and flexibiiity of the principal, trainer, 
and professor have helped the teachers 
move from a positkm of resistance to 
one of involvement 

Chattering in relaxed tones, the 
teachers move into the hallway. The 
training session has offered them a 
rare opportunity to talk about problems 
and common experiences in teaching. 
They share last-minute information 

before climbing Into their cars, again 
as tooiated as In their separate class- 
rooms. 

Meanwhile, in the neariy deserted 
boardroom, the trainer and the profes- 
sor review the morning's dtocusston. 
"fhey are pleased The teachers' re- 
<iuest to meet again has offered unex- 
pected conflrmatkm that the experi- 
ment seems to be woridng. The assis- 
tant superintendent stops by brief^ to 
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say that the principal conakters the 
setakNi a dear success» 

Despite barred windows, empty 
chain, and a aeriaa of tables littered 
with empty coffee cups, the boardroom 
radiates warmth. This Is colleglalHy on 
a level unknown to pigeons. 
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A Checklist for Observing Teachers' Use 
of the Missouri "Active Teaching" Model In Mathematics 
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APPENDIX F 



A ClMckliit for O ueflrri ng TMehtn* Use 
of the MiMOMri "Aetivo Teaehing" Model in Metheaetiei 

uiti Teacher Observation # 

Place a checkmark < ) in the blank next to all statements that hold true. 

Deilf Review 

reviewed the concepts and skills associated with the homework 

collected and dealt with homework assignments 

asked several mental computation exercises 

spent a total of about 8 minutes in doing all of the above* 

DeretopMent 

briefly focused on prerequisite skills and concepts 

focused on meaning and promoting student understanding by: 

explaining the processes used to reach a particular solution or answer 

using concrete illustrations and examples 

using manipulatives 

assessed student comprehension 

used process/product questions 

used controlled practice 

repeated and elaborated on the meaning portion as necessary ' 

spent a total of about 20 minutes on the development phase of the lesson 

Seetwork 

provided uninterrupted practice 

got everyone involved 

continuously monitored student's work and provided feedback to individuals as 

needed 

let students know that their work would be checked at the end of the period 

checked students' work at the end of the period 

spent a total of about 15 minutes on seatwork 

HoDeworlL Aaigiuieiit 

assigned homework 

♦ It is intended that on Mondays about 20 minutes will be allocated for reviewing 
concepts and skills covered during the previous week. 
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APPENDIX G 

Time Line Observation Form for Recording 
Good and dirouwa* Variables 
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Observation Form 

Teacher's Nam e 

Your Name 

Dat e 

01 s t r1 ct 

No. of Students In Class Today 

Time Math Class Started 

Before class starts, ask teacher: 

1. What will your math lesson be about today? 

2. Will you be reviewing any of yesterday's work? 

3. Last time did you assign any homework to be done out of class? 
If "yes," will you do anything with the homeworlr ioday? 

# 

4. Will your students be doing any seatwork today? If "yes," what are 
acceptable things for the children to do If they finish early? 

I 

5. Will any of the children be leaving during the class today? 

6. Will today be a typical math lesson for you? 

18a 
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M4-83 

Post Observation Form 

1. Student comprehension when teacher explained concepts and procedures. 

Students followed teacher's explanation with no confusion. 

Good comprehension, with occasional confusion. 

Lots of confusion. 

Not applicable 

2. Student comprehension during seatwork. 

Students worked independently all or most of the time. 

Students occasionally asked for teacher assistance. 

Heavy demand for teacher assistance. 

Not applicable. 

3. Were word problems included In the lesson? 
No. 

Yes, for 2 minutes or less. 

Yes, for more than 2 minutes. 

4. Notes on classroom organization and unusual occurrences. 
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APPENDIX H 

Manual for Scoring Good and Grouws* Variables 
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MANUAL FOR SCORING GOOD AND GROUWS VARIABLES 
Definition of Variables 

Transition (T) 

Transition refers to the process of going from one phase of the class period 
to another. Transition also Involves (a) the period of time from when the class 
Is scheduled to begin and when It actually gets productively underway; (b) when 
the teacher signals end of class Is approaching and students start putting 
things away before the lesson has actually ended; and (c) the distribution or 
collection of seatwork, homework, and other materials. 
Interruptions (I) 

Interruptions Include such events as fire alarms. Intercom messages, and 
classroom discipline problems. 
Review of Previous Work and Quizzes (R) 

Review refers to work on old material. It deals with concepts and skills 
that the students have learned prior to the lesson being observed. The concepts 
and skills can be from the previous day or from much earlier. Use of games, 
puzzles, worksheets, etc., which are used to review concepts and skills fall 
into this category. Review of previously-taken quizzes and tests and giving of 
quizzes are coded as Q. 
Checking of P rior Homework (C) 

This variable is similar to R, but is limited to review of homework pre- 
viously assigned. Tht "checking" process can include in-class grading of 
homework, the showing of solutions to homework problems, and answering of 
questions about the homework. 
Assignment of Homework (A) 

This variable is an index of the time spent giving an homework assignment. 
It Includes such activities as distributing homework problems and giving 
homework directions. 
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This variable Is the time spent giving a quiz In class. 
Mental Computation (M) 

This variable refers, to time spent on computation without the aid of paper- 
and-pencll, blackboard, or calculators. The computations are done mentally. If 
the computations are not done mentally, code as "Seatwork" or "Controlled 
Practice Within Development." The problems that students are asked to solve 
mentally may refer to new or old computational skills. 
Development (D) 

The development part of a math lesson Involves helping students comprehend 
new concepts and skills. Development activities often Include teacher explana- 
tions and demonstrating use of manipulative materials, making o^ -omp-^'l-ons, 
search for patterns, class discuslon, group work, and use of audio-visual 
materials. Brief review of prerequisite concepts and skills may occur, but this 
activity Is counted as time spent on development If Its purpose Is to facilitate 
the development part of a lesson rather than to review previous work (R). 
Controlled Practice Within Development (P) 

This variable Includes controlljd practice In which one or two problems are 
stated and then Immediate feedback Is given on the correctness of the responses. 
Controlled practice Is generally used to help the teacher Identify and correct 
student misunderstandings just before seatwork to make sure the students can do 
the seatwork successfully. 
Directions for Seatwork (S) 

This variable refers to time spent on giving directions to students on how 
to do their seatwork. Time spent by students getting ready to do seatwork 
(e.g., getting materials ready) should be coded as a transition (T), 
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Seatwork (+ or -) 

Seatwork refers to practice on concepts and skills covered In the day's 
lesson. It may Include some review of previous work. If the problems are 
entirely a review of previous work, though, the activity should be coded as 
"Review of Previous Work" (R). If the teacher Is monitoring the seatwork or 
assisting students during the time period being observed. It Is coded as "+". 
If the teacher Is not doing either of these activities. It Is coded as "-". 
Checking of Seatwork at End (E) 

This variable Includes such activities as the teacher alerting students that 
their seatwork will be checked, reading correct answers to the seatwork 
problems, or asking students to check their answers in some way. 
Number of Students Off Task 

A student is on task if he is engaged in an activity assigned by the teacher 
or that is academically appropriate to the situation. Examples of on-task acti- 
vities are: listening to the teacher, doing seatwork, answering the teacher's 
questions, working at the blackboard. Examples of off task activities are: 
chatting with another student about a nonacademic matter, waiting for the 
teacher's help during seatwork, daydreaming, standing around, waiting for the 
lesson to start or end. 

Decision Rules for Observing Good and Grouws' Variables 
1. A math lesson begins when the teacher signals that it has begun. Some 
students may not be at iheir seats or even in the room. Some students may 
be at their desks, but are not ready to begin their math work. Code the 
interval between the teacher's signal and students' setting down to work 
as TRANSITION. 



ERIC 



202 

191 



2. A math lesson ends when the teacher Indicates that 1t has ended* For 
example, "put your math books away now." Do not code any TRANSITION time 
. after the end-of-lesson signal has occurred* 

3* All of the codes except SEATWORK are reserved for whole group Instruction* 
Individualized math Instruction Is to be coded as SEATWORK, even when the 
teacher works with a small group of students or with several small groups. 

4* If a student teacher Is In the classroom, do not code anything that he or 
she or his or her students are doing. Do not count the students whom the 
student teacher Is Instructing as part of the total number of students In 
the class. Also, do not code these students as on or off task. However, 
do describe this situation In your post-observation notes. 

5. Code REVIEW (R) only when the teacher Is reviewing the prior day's work. 
The review should be In the form of a summary of what was studied the 
previous day. If the review refers to an unspecified time frame or a time 
frame other than yesterday. It Is likely that DEVELOPMENT (D) Is occurring. 
If the teacher starts with a review of yesterday's work, but proceeds to 
reteach the content, start by coding REVIEW and then switch to DEVELOPMENT. 

6. CHECK PRIOR HOMEWORK refers specifically to checking the answers to 
homework. This code applies to the teacher giving or asking for the correct 
answers to the homework. It also applies to stating or asking how the 
answers were obtained. If you are unsure whether the teacher Is checking 
homework op seatwork, ask the teacher to clarify when the lesson Is over. 
Change your coding If necessary. 

7. CHECK SEATWORK AT END (E) will usually occur at the end of a lesson. 
However, It can also occur at the beginning of the lesson If the teacher 
goes over the previous day's seatwork. If the teacher collects seatwork 
without saying anything, ask teacher after lesson If she Intended to check 

the seatwork and give feedback to the students; If yes, circle two minutes 
of Es. 
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8. Use the ASSIGN HOMEWORK (A) only 1f homework Is assigned to be done out 
of class. A teacher may assign homework, but allow students to complete It 
In class. The ASSIGN HOMEWORK code should only be used If there Is an 
expectation that the students w111 be doing math work outside of the class 
or some time later after the math lesson Is ended. Ask the teacher what 
his or her Intentions are If you are not certain. 

9. MENTAL COMPUTATION refers specifically to quick computations that the 
student does In his or her head. The student can write the answers on paper, 
but he or she should not do the computations on paper. If the computations 
are done on paper. It Is likely that the QUIZ code should be used. 

10. CONTROLLED PRACTICE should be used only when: the teacher assigns one or 
a few problems to be done; the teacher checks the accuracy of students' 
answers right away; the problems occur near the end of the development 
period (unless students can't do the problems and the teacher needs to do 
further development); and the problems should be of the same type as 
will appear In seatwork. All of these conditions should be present for 
the CONTROLLED PRACTICE (P) code to be used. 

11. TRANSITION (T) occurs whenever students are waiting for something to occur 
(e.g., waiting for the teacher's help), or are getting ready to do 
something. 

12. Students are OFF TASK whenever they are not engaged In Instructional 1y 
relevant behavior that meets the teacher's expectations. 

13. Whole class Instruction occurs only when teacher is working with at least 
three- fourths of the class In a group activity. Otherwise It Is seatwork. 

14. If teacher Is Instructing a small group of the students (less than one- 
fourth) while other students are doing seatwork, count as S-. The exception 
Is If teacher stops working with subgroup and assists a seatwork student 
(code S+). 

193 204 



15. If you observe a teacher on Friday, ask the teacher after class whether 
she normally assigns homework. If yes, code one homework Interval^ 

16. Ask the teacher after class whether she collected and checked homework 
earlier In the day. (Do this only If no homework collection or checking 
occurred.) If yes, have teacher estimate time and fill In appropriate 
number of "check prior homework" Intervals. 

17. If the teacher Is engaged In "T" or "I" behavior while you are doing an 
"off task" tally, count all students as off task because they are not 
engaged In Instructional ly relevant behavior. 



ERIC 



205 

194 



APPENDIX I 

Recloaale €c<r Tine Criteria for Behavior Categories 
1a Cood and Grouws* Instructional Model 
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APPENDIX I 

Rationale for Time Criteria for Behavior Categories 
In Good and Grouws' Instructional Model 

The rationale for the time criteria used to analyze the data from the 

experiment was derived primarily from the Teachers Manual for Good and Grouws' 

tralnTng^ program. The Teachers Manual Is Included In a report by Good and 

Grouws (1981). The essential behaviors described In the manual are summarized 

In Table 2-2. 

Quoted material In the following sections Is from the Teachers Manual, using 
the pagination of the Good and Grouws (J981) report. 

1. TRANSITIONS - 2 minutes or less 

The need to minimize transitions Is only mentioned specifically with respect 
to seatwork. "To help optimize the effectiveness of seatwork, three general 
principles should be observed... The first principle, momentum , ...meanCs] 
keeping the ball rolling without any sharp break In teaching activity and In 
student Involvement" (p. 103). The need to minimize transitions Is also Implied 
In the section on Instructional pacing (p. 120). 

The Teachers Manual states no time criteria for transitions. Since some 
transitional t'^me seems Inevitable, the decision was made to use the pretreat- 
ment mean 9S a minimal criterion. This mean Is 1.94 minutes (see table 4.2), 
and was rounded off to 2 minutes. 

2. INTERRUPTIONS - 0 minutes 

Although Interruptions are not discussed explicitly In the Teachers Manual, 
the desirability of avoiding them Is implied In the discussion of Instructional 
mcnentum (p. 103) and pacing (p. 120). Since It seems possible to avoid 
Interruptions completely with careful Instructional planning, 0 minutes was set 
as the criterion. 

3. REVIEW PREVIOUS WORK - 1 minute or more 

The Teachers Manual recommends "1-2 minutes on review" (p. 122). 
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4. CHECK PRIOR HOMEWORK - 2 minutes or more 

"... It Should take only a couple of minutes to check homework" (p. 121). 
"In general, we think the following situation will be most applicable... 3«4 
minutes checking homework" (p. 122). The more modest figure of 2 minutes 
Implied by "couple of minutes" was used as the criterion for this behavior 
category. 

5. ASSIGN HOMEWORK - 1 second or more 

The Teachers Manual does not contain a recommendation on the amount of time 
that assigning homework should take. It Is only Important that the behavior 
occur, and this Is reflected by the "1 second or more" criterion. 

6. QUIZ - 3 minutes or less 

This behavior category Is not mentioned In the Teachers Manual, except In 
the section on cumulative monthly reviews (p. 115). It Is mentioned there as 
Just one of several methods of review. This category was Included In the 
classroom observation Instrument, however, because It occurred frequently In 
pilot observations of teachers' lessons. The mean length of this category In 
the pretreatment lesson was 2.94 minutes, and this score (rounded to 3 minutes) 
was used as the criterion. It seemecl Implicit In Good and Grouws' model that 
behaviors like quizzes should not dominate a math lesson. A quiz longer than 3 
minutes might leave less time for other, more Important behaviors advocated In 
their Instructional model. 

7. MENTAL COMPUTATION - 3 minutes or more 

The Teachers Manual states, "We would like for you to Include 3-5 minutes on 
mental computation activities each day..." (p. 117). 

8. DEVELOPMENT - 5 minutes or more 

The summary table on page 94 In the Teachers Manual (reproduced in the pre- 
sent report as table 2- ) recommends "about 20 minutes" for development. 
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However, the data from the present experiment suggests that this time criterion 
Is unreal Istlcally high. In fact, the treatment Implementation analysis In Good 
and Grouws' fourth-grade study Includes the percentage of teachers who spent "at 
least five minutes on development" (see table of the present report). This 
lesser time criterion was selected for use in the present study, 

9. CONTROLLED PRACTICE - 1 minute or more 

The Teachers Manual does not make a specific time recommendation for this 
behavior category. There Is mention that controlled practice (defined In the 
present study as the assignment of practice problems) and oral questioning "can 
be completed In 3-5 minutes" (p. 97). Controlled practice can be very brief If 
students have understood the teacher's prior demonstration. Therefore, a mini- 
mal time criterion of "1 minute or more" was used to Indicate satisfactory use 
of this behavior. 

10. DIRECTIONS FOR SEATWORK - 1 minute or less 

The Teachers Manual states that the teacher should initiate seatwork "with a 
simple and direct statement" (p. 103). The treatment Implementation analysis In 
Good and Grouws' fourth-grade study Included a variable described as, "Did seat- 
work directions take longer than one minute?" 

11. MONITORED SEATWORK - 15 minutes or less 

The Teachers Manual states, "...we recommend that about 10-15 minutes each 
day be allotted for seatwork. Ten to fifteen minutes allows sufficient time for 
students to work enough problems to achieve Increased proficiency but not so long 
as to bring about boredom, lack of task Involvement... If practice time does not 
exceed 15 minutes, few students are likely to be bored" (p. 102). 

12. UNMONITORED SEATWORK - 2 minutes or less 

The Teachers Manual does not state explicitly that teachers should avoid 
unmonltored seatwork. It Is strongly Implied, however. In the discussion of 
seatwork (pp. 101-106). It seems unrealistic to expect teachers to monitor 
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seatwork one hundred percent of the Vns, lo a less strict criterion was used. 
The decision was mad^ to use the lowest mean amount of unmonltored seatwork 
achieved by a treatment group In the pre-observatlon. This mean was 2.39 minu- 
tes, achieved by the regular Inservlce group In Districts 2-3. This figure was 
rounded to 2 minutes In determining the time criterion. 
12. CHECK SEATWORK AT END - 1 minute or more 

The Teachers Manual states, "At the very end of the seatwork period, hold 
students accountable for their work by asking Individual students to give the 
answer to a few of the assigned problems. This checking of answers should be 
very rapid and you need only check 3 or 4 of the problems..." (pp. 105-106). A 
more specific time criterion Is not stated. Therefore, the decision was made to 
use the pretreatment mean as a minimal criterion. The actual mean was 1.27 
minutes, which was rounded to 1 minute to provide the criterion. 
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MJuitad Pott m M««d TtMtMiK »tos «ar tmtnctlonil Bahwlori ty Dtttrkt 







TrtiMnt MMi Miuit«i ty Pre Scores on Corresponding Virlibit 
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TABLE A-2 

Number of Techniques Used to Criterion Level 
by Each Treatment Group In Each District 



District 1 





Principal Involvement 
(N-9) 


Regular Inservlce 
(N-10) 


Control 
(N-10) 




M 


(SD) 


N 


(SD) 


M (SD) 


Pre 


4.33 


(1.32) 


4.30 


(2.36) 


5.00 (1.56) 


Post 


7.00 


(2.69) 


6.40 


(2.17) 


5.50 (1.84) 


Delayed 


8.11 


(2.09) 


6.90 


(1.91) 


4.90 ( .74) 






Districts 2- 


3 




* 


Principal Involvement 
(N-9) 


Regular Inservlce 
(N-7) 


Control 
(N-8) 




N 
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M 
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M (SD) 


Pre 
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8.44 
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8.29) 
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5.88 (1.96) 
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TABLE A-3 

Ptromtagt of Ttacher^ Using Instructional 

Behaviors for Criterion Time Periods 



Principal 
Involvenent 







(N-9) 


Transitions 

(2 minutes or less) 


Pre 

Post 

Del^ 
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33X 
IIX 
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(no Interruptions) 


Pre 

Post 

Delayed 


44X 
44% 
33X 
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Pre 

Post 

Del^iyed 


IIX 
67X 
78X 


Quiz 

(3 minutes or less) 


Pre 

Post 

Delayed 


89X 
lOOX 
lOQX 


Mental Corputatlon 
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I Districts 2-3 
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TABLE M 

Ungth In Tin* (In-Mnutts) for Instructional 
Behaviors ly Tyvatftent 6roup Within Districts 



Instructional Behaviors 





District 1 
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rnncipBi 




Control 


Or*A nal 

rnncipai 




Control 




TnwolwiifiMtk 
















fNslO) 

VII^aW J 


(N40) 










M 


M 


M 


M 




• 

M 




(SD) 


(SD) 


(SD) 


(SD) 


(SO) 


(SO) 


Pre 


1.24 


2.49 


1.79 


2.28 


2.04 


1.79 








(1.46) 


^1 41^ 




(1.59) 


Post 


3.32 




2.03 


3.39 


2.20 


2.29 








(1.89) 




f2.13^ 


(1.61) 


Delayed 


2.96 


2.87 


3.26 


2.84 


3.51 


1.41 


(1.35) 


(2.21) 


(3.60) 


(2.86) 


(1.60) 


(1.31) 


Pre 


.61 


1.06 

Ae w 


1.09 


.58 


.29 


1.58 




( 7(\\ 


^leoo; 


(2.04) 






(2.06) 


Post 


1 39 


.17 


.31 


.28 


.21 


•59 








(.72) 






(.69) 


Delayed 


.71 


el8 


.87 


.33 


.97 


.60 


(1.02) 


(•30) 


(1.39) 


(.56) 


(1.85) 


(1.19) 


Pre 


.75 


.70 


2.18 


2.71 


3.43 


.94 








(4.05) 


/4 14) 


^3 36^ 


(1.32) 


Post 


A 70 


1 20 


6.78 


2.37 


4.74 


6.85 








(11.36) 




f6 04) 
\w.i^y 


(8.08) 


Delayed 


2.14 


1.48 


.00 


1.92 


.90 


2.25 


(2.17) 


(1.51) 


(.00) 


(2.90) 


(1.14) 


(3.92) 


Pre 


nn 


20 


1.15 


.50 


.07 


.00 








(2.43) 






(.00) 


Post 


CeOiL 


4 83 


.00 


4.12 


4.31 


.00 






\c,yt } 


(.00) 


^3 16) 


^4.97^ 


(.00) 


Delayed 


1.87 


2.70 


1.33 


3.36 


4.87 


.00 


(1-44) 


(2.83) 


(4.03) 


(3.11) 


(5.26) 


(.00) 


Pre 


•WW 


.15 


.06 


.00 


.07 


.25 








(.18) 




L19^ 


(.46) 


Post 


31 


.39 


.05 


.71 


1.49 


.00 








(.16) 




Q.60) 


(.00) 


Delayed 


.43 


.43 


.00 
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(4.75) 
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(4.24) 


(2.32) 


(3.09) 


Post 


.00 


3.38 


1.50 


.89 


.00 


3.06 




(.00) 


(4.90) 


(4.74) 


(2.67) 


(.00) 


(6.21) 


Delayed 


.00 


1.03 


4.29 


L50 


3.21 


.88 


(^00) 


(1.98) 


(10.68) 


'(3.28) 


(8.50) 


(1.94) 



Transitions 



Interruptions 



Review Previous kbrk 



Check Prior Hcmework 



Assign Hanework 



Quiz 



ERIC 



204 



215 



Table M Continued 



Mental Ccmputatlon 



Development 



Controlled Practice 



Pre .00 
(.00) 

Post 1.87 
(2.35) 

Delayed 1.50 
(1.77) 

Pre 6.53 
(10.56) 
Post 4.19 

(4.35) 
Delayed 16.32 

(8.73) 

Pre .00 
(.00) 
Post .76 

(1.53) 
Oel^iyed 1.67 

(2.33) 



Directions for Seatwork Pre 
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Pre 
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.37 
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(1.95) 



.16 


1.68 


•70 


1 7A 


1 fiC 


(.41) 






(^•^; 




2.03 


AC 

•25 




A 70 
4«/S 


•oi 
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(2.87) 


/I e9\ 
(1.52) 


/I 11 \ 
(1«11) 




/I 70) 


4 A^ 

1.86 


.86 


.77 


i 10 

ulo 


oc 
•o9 
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(.52) 
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(1.45) 


(1.02) 


(.26) 
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TABLE A-5 

Descriptive Statistics for Curriculum-Referenced Test 
by Treatment Group Within District and Grade Level 



District and 
Grade Level 



District 1 
Grade 4 



District 1 
Grade 5 



Districts 2-3 
Grade 4 



Districts 2-3 
Grade 5 



Principal Involvement 



"Pre 
Test 



PosT 
Test 



33.76 42.18 
(7.52) (10.77) 
Gain » 8.42 



25.76 34,08 
(9.37) (7.19) 
Gain « 8.32 



38.48 48.41 
(5.16) (1.15) 
Gain » 9.93 



37.78 42.22 
(8.36) (8.99) 
Gain - 4.44 



Regular Inservice 



T 



Tre 
Test 
TOT 



Post 
Test 
TCTT 



36.98 43.61 
(7.78) (9.58) 
Gain » 6.63 



4 38.33 40.53 
(5.53) (7.19) 
Gain » 2.20 



42.44 49.64 
(8.49) (13.28) 
Gain » 7.20 



42.03 47.45 
(6.38) (7.74) 
Gain - 5.42 



Control 



T 



Pre 
Test 



ToiT 
Test 



39.41 48.50 
(4.71) (2.50 
Gain « 9.09 



37.82 42.42 
(5.58) (7.27 
Gain » 4.60 



40.54 44.81 
(5.84) (4.80 
Gain- «,4.27 



36.24 40.18 
(2.09) (2.08 
Gain • 3.94 
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TABL£ A-6 

Descriptive Statistics for Standardized Achievement Test (Total Score) 
by Treatment Group Within District and Grade Level 



District and 
Grade Level 



District 1 
Grade 4 



District 1 
Grade 5 



Districts 2-3 
Grade 4 



Districts 2-3 
Grade 5 



Principal Involvement 
Post — 



Pre 
Test 

imr 



Test 



: 48.20 54.31 
(1.89) (1.51) 
Gain - 6.11 



44.82 52.68 
(8.12) (9.63) 
Gain • 7.86 



50.13 57.88 
(4.47) (1.31) 
Gain « 7.75 



51.46 56.65 
(13.22) (14.51) 
Gain « 5.19 



Regular Inservlce 



Pre t*ost 
Test Test 



51.56 65.16 
(5.29) (5.90) 
Gain > 13.60 



50.53 59.24 
(6.27) (3.98) 
Gain « 8.71 



49.48 59.62 
(5.04) (2.5) 
Gain » 10.14 



55.66 55.75 
(3.46) (4.91) 
Gain ■ .09 



Control 



Pre 
Test 

TOT 



f'ost 
Test 



53.09 64.73 
(4.60) (4.26) 
Gain * 11.64 



51.01 58.23 
(6.69) (5.19] 
Gain > 7.22 



52.42 51.94 
(2.78) (5.77] 
Gain - -.48 



44.27 
(9.53) 
Gain - 4.00 



48.27 
(2.78) 
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TABLE A-7 

Descriptive Statistics for Standardized Achievement Test 
(Computation Score) by Treatment Group Within District and Grade Level 



District and 
Grade Level 



District 1 
Grade 4 



District 1 
Grade 5 



Districts 2-3 
Grade 4 



Districts 2-3 
Grade 5 



Principal Involvement 



IT 



Pre 
Test 



Post 
Test 



Ml 



45.45 51.85 
(4.07) (1.82) 
Gain * 6*40 

40.96 51.27 
(/.61) (12.21) 
Gain » 10.31 

47.35 58.03 
(5.06) (2.44) 
Gain 10.68 

51.67 56.21 
(12.30) (13.00) 
Gain - 4.54 



Regular Inservlce 



T 



Pre 
Test 



Post 
Test 



5 50.54 68.46 
(6.45) (7.31) 
Gain " 17.92 

4' 46.58 60.94 
(6.72) (3.96) 
Gain « 14.36 

3 47.19 60.74 
(6.08) (3.99) 
Gain " 13.55 

4 52.92 52.29 
(4.16) (5.13) 
Gain « -.63 



Control 



T 



Pre 
Test 



Post 
Test 



53.12 67.93 

(5.21) (6.96] 

Gain > 14.81 

49.68 59.35 

(7.13) (5.90; 

Gain « 9.67 

51.66 51.12 

(2.85) (6.86 

Gain * -.54 

44.78 51.06 

(8.99) (2.49; 

Gain > 6.28 



JRJC 
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TABLE A-8 

Descriptive Statistics for Student Off-Task Behavior 



Kre 



Post 



District 1 
Principal Regular 



Involvement 
(N-9) 



M 
M 

(SD 



0#!]ayed H 
(SD 



5.44 

(2.77) 

3.51 
(2.57) 

3.62 
(2.82) 



In service 
(N-10) 



Control 
(N-10) 



Principal 
Involvement 
(N-9) 



Districts 2-3 
Regular 



AVERAGE NUMBER OF 
OFF-TASK STUDENTS 



4.84 
(2.68) 

4.15 
(3.20) 

3.50 
(1.71) 



3.73 
(2.18) 

3.31 
(1.76) 

6.58 
(4.50) 



2.51 
(2.79) 

1.83 
(1.83) 

3.71 
(2.28) 



AVERAGE PERCENTAGE 
OF OFF-TASK STUDENTS 



In service 
(N-7) 



3.35 
(2.12) 

2.44 
(1.90) 

3.23 
(1.55) 



Control 
(N-8) 



4.59 
(3.05) 

4.32 
(1.78) 

3.08 
( .68) 









NUMBER OF STUDENTS 
IN CUSS 


« 




Pre 


H 

(SD) 


22.06 
(2.30) 


22.60 
(3.17) 


20.50 
(2.22) 


18.56 
(8.62) 


21.29 
(3.30) 


21.50 
(6.72) 


Post 


M 

(SD) 


21.72 
(4.54) 


22.40 
(3.27) 


19.50 
(2.46) 


18.17 
(7.75) 


19.71 
(4.19) 


20.94 
(5.97) 


Delayed 


M 

(SD) 


20.44 
(3.54) 


23.00 
(3.30) 


19.00 
(2.92) 


18.67 
(7.28) 


19.29 
(3.64) 


21.63 
(7.07) 



Pre 


M 


24.89 


21.20 


18.40 


12.00 


15.29 


20.63 


(SD) 


(13.20) 


(9.96) 


(10.88) 


(10.67) 


(8.94) 


(12.76) 


Post 


M 


17.22 


18.70 


17.50 


10.22 


12.86 


22.00 




(SD) 


(13.79) 


(15.30) 


(9.98) 


(8.93) 


(10.53) 


(9.80) 


Delayed 


M 


18.00 


15.00 


33.50 


19.78 


17.57 


15.63 


(SD) 


(13.74) 


(6.70) 


(21.28) 


(10.57) 


(9.68) 


(5.55) 
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